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ABSTRACT 

Sensitivity data is defined as involving two 
response categories, with responses observed at different levels of 
some variable. The responses are taken to indicate sensitivity to the 
variable and iray be labeled "positive" or "negative." The countback 
method offers confidence limits for the 50% point, the level of the 
variable for which the two responses are equally likely, in the 
special case when cnly cne response is observed at each level. A 
lcwer bcund cn the confidence coefficient for these limits can be 
given, assuming cnly independent responses and the existence of 
something like a 50% point. No specific model of response 
probabilities is required, and the variable need not even be 
continuous. Tc gain a greater understanding of the countback method, 
it is used here tc analyze sensitivity data obtained in an experiment 
on the perception of depth in pictures, and to study two artificial 
situations where the response probabilities can be specified exactly- 
The ccuntback method is compared with the minimum Chi-square and the 
maximum likelihood estimation methods and its extension to data with 
more than cne response per level is discussed. The results suggest 
that the countback method has a significant place among previous 
methods for analyzing sensitivity data. Appendices include 
historical, statistical, simulation, and experimental details. 
(Author/CK) 
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THE COUNT BACK METHOD FOR ANALYZING 



SENSITIVITY DATA 
Abstract 

Sensitivity data are defined as involving two response categories, 
with responses observed at different levels of some variable of interest. 
The responses are taken to indicate sensitivity to the variable, and, 

in this regard, may be labeled "positive" and "negative." Of particular 

✓ 

interest in analyzing sensitivity data is the estimation of the 50$ 
point, the level of the variable for which the two responses are equally 
likely. 

The countback method provides confidence limits for the 50$ point 
in the special (but important ) case when only one response is observed 
at each level. A lower bound on the confidence coefficient for these 
limits can be given, assuming only independent responses and the exis- 
tence of something like a 50$ point. No specific model of response 
probabilities is required, and the variable of interest need not even 
be continuous . 

To gain a greater understanding of the countback method, we use it 
to analyze sensitivity data obtained in an experiment on the perception 
of depth in pictures. Its performance is also studied in two artificial 
situations, where the response probabilities can be specified exactly. 

In the first of these, the probability of a positive response at 
different levels is given by the cumulative logistic model. The second 
situation is highly asymmetric, using the logistic model below the 50$ 
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point and an errorless model (the positive response probability set 
equal to l) above it. 

For purposes of comparison, two asymptotically efficient estima- 

2 

tion methods, namely maximum likelihood and minimum X , are also 
applied to the cases described above. In the two artificial situations, 
where actual confidence coefficients can be determined, both of these 
methods fall short of their asymptotic performance levels. 

The countback confidence limits, by contrast, perform at a sub- 
stantially higher level than that given as a lower bound by the theory. 
Moreover, the ability of conservatively chosen countback l imi ts to 
reject null hypotheses is quite adequate for the purposes of the 
experiment mentioned earlier. Finally, a simple extension of the 
countback method to data with more than one response per level is 
discussed. 

On the basis of these results, the countback method is seen as 
taking a significant place among the previously existing methods for 
analyzing sensitivity data. 

Appendices are provided for those interested in statistical, 
simulation, experimental, and historical details. 
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THE COUNT BACK METHOD FOR ANALYZING 



SENSITIVITY DATA 

Consider the following experiments, (l) A human subject is asked 
to observe very dim flashes of light. The intensity of the flash is 
varied and the subject reports, for each flash, whether or not he was 
able to see it. (2) A weight is dropped from different heights onto 
several samples of the same explosive. In each case, it is recorded 
if the sample exploded. (5) Insects of a given species are exposed to 
different concentrations of an insecticide. Each is observed to see if 
it dies as a result, (k) Several different dilutions are made from a 
sample of soil. The presence or absence of a species of protozoan in 
a fixed amount of each dilution is determined after exposure to a 
suitable medium. 

These are examples of what may be called the sensitivity experiment. 
This is most clearly an apt label for the first three . They are aimed, 
respectively, at determining the sensitivity of the eye to light, of 
explosives to impact, and of insects to insecticide. In each, a two- 
category response is examined to see the effect of different levels of 
the variable of interest. Each has the goal of estimating the value 
of the variable for which the two responses will occur with equal 
probability. This value (which will be referred to as the 50$ point) 
is taken in these examples to be a good indication of sensitivity. 

It should be noted that this probabilistic approach is an essential 
feature of the sensitivity experiment, and for good reason. Only rarely 
do sensitivity data indicate the existence of a critical level, with 
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one response always observed below and the other always observed above 
it. Consequently, the simple notion of a critical level had to be 
generalized, and the 50# point is the most widely accepted generalization. 

In the fourth experiment, a model is used to estimate directly the 

ft 

number of organisms in the original sample, but this value is a simple 

function of the dilution at which 50# of the exposed mediums will be * 

sterile. In the same way, the first experiment may be concerned with 

the minimum number of photons necessary for vision, rather than the 

intensity which produces a visible flash half the time. Again, these 

two are directly related when an appropriate model is introduced. 

Thus, the general sensitivity problem, as it has been called, is 
to estimate a 50# point, or some function of it, from two-category 
response data. 

There is some latitude available in designing a sensitivity 
experiment. In many cases, for instance, the experimenter is free to 
choose the levels of the variable as well as the number of observations 
at each level. He may even be free to make these decisions as the 
experiment progresses. 

In general, the choice of design is closely related to the contem- 

i 

plated analysis. Consider three different methods of analysis. One 
widely adopted method, known as probit analysis (Finney, 1952), works > 

well when a large number of observations is made at each of a limited 
number of closely spaced levels, particularly if these happen to cover 
the 50# point. Fisher's (1922) approach to dilution series is suggested 
for use with a moderate number of dilutions and relatively few observa- 
tions at each. It requires, however, that successive dilutions be in 
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a constant ratio throughout the series. The up-and-down method of 
Dixon and Mood (1948) requires the levels to he chosen in advance, hut 
lets the number per level he determined hy the experiment. As they 
describe it for testing the sensitivity of explosives to impact, if a 
sample tested at one level explodes, a new sample should he tested at 
the next lower level. If there is no explosion, the new test should he 
made at the next higher level. (Examples of the use of these methods 
are listed following the Bibliography.) Finney (1964, esp. pp. 551-553) 
provides recommendations on additional methods, not mentioned above. 

There will, of course, be many legitimate examples of the sensi- 
tivity problem for which none of the above experimental designs is 
desirable, or even possible. A particularly striking case is a variation 
of the insecticide experiment described above. If the researcher is 
concerned with the effective level of insecticide involved, he may 
measure the quantity in each insect at the end of the experiment. Thus, 
he has no more than a crude control over the levels of the variable, 
and only a small chance of making more than one observation at any of 
the levels attained. In such a case, one of the methods mentioned 
above, namely probit analysis, can still be applied (see Bliss, 1938; 
Finney, 19^7 > 1952, sec. 43), but the results should be treated with 
caution, as we will see later. 

More generally, a sensitivity experiment of a preliminary nature 
may require wide coverage with a minimal number of observations involved. 
Consequently, a method of analysis specifically designed to deal with 
one observation at a level and allowing arbitrarily spaced levels would 
fill a significant gap left open by the collection of techniques 
referred to above. 
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There is another limitation on the previously mentioned approaches, 
namely that the variable of interest is assumed to be continuous. This 
makes it easy to assume a continuous effect of the variable on the 
response probabilities, and, with a little more effort, the existence 
of an exact (and unique) 50$ point. There is, however, nothing in the 
design of the sensitivity experiment that prohibits the -use of an 
essentially discontinuous variable. Since only discrete values are 
used, all that is required is for the variable to have a progressive 
effect on response probabilities. One example from psychology is an 
experiment on memory span. The subject hears a list of digits read and 
is asked to repeat the list. The variable here is the length of the 
list. The two possible categories of response are a repetition and an 
error. In this experiment, it is very unlikely that a 50$ point 
exists. Nonetheless, something like a 50$ point would be very useful 
in summarizing the data. 

Since the lengths are labeled numerically, it would be easy to 
pretend that a continuous variable exists , and then apply some standard 
technique. Statements such as ’'the subject was able to repeat a list of 
6.76 digits with probability . 5 ” could then be expected to follow 
routinely. There is, however, an approach which does less violence to 
one's sensibilities, namely, the use of confidence limits. 

An important part of the sensitivity problem is indicating the 
quality of the estimate. An accepted means of so indicating is with 
confidence limits . Confidence limits are calculated from the data in a 
way which guarantees the value being estimated will fall between them 
with some given probability (.95 is a favorite value) upon successive 
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repetitions of the experiment. This does not mean that the true value 
has different locations with different probabilities. It is considered 
to be fixed. Instead, it means that the confidence limits change with 
each repetition of the experiment and that 95$ of the time (for 95$ 
limits) the true value will fall between them. Moreover, this guarantee 
holds regardless of the location of the true value. 

Even when there is no true value, as in the last example, it may- 
still be possible to find modified confidence limits. Suppose, on the 
basis of memory span data we have collected, we choose two list lengths, 

L and U . L is chosen so that it is likely the subject could repeat 
lists of length L or shorter more than half the time. U is chosen so 
that it is unlikely the subject could repeat lists of length U or longer 
more than half the time. If "likely" and "unlikely" were formalized 
satisfactorily, L and U could then be considered as confidence limits 
for the 50$ point, even though no 50$ point can be realized. A relevant 
statement about the experiment then might be: "With probability .95> 

errors predominate for lists of ten or more digits, while a repetition 
is the more likely event with lists of three or fewer digits . " 

A framework has now been provided for a new approach to the sensi- 
tivity problem. This approach, to be described below, is designed to 
handle one observed response per level of a possibly discrete variable, 
and to deal directly in terms of confidence limits for the 50$ point. 

For reasons which will become clear, we c all it the countback method. 

Returning to the first example, suppose that one response has been 
obtained for each of several intensities of the light flash. The first 
step is to single out the lowest level at which the flash was seen and 
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the highest level at which it was not. These will be referred to as the 
low and high levels . 

At this stage, we could take the average of these two levels and 
use the result as an estimate of the 50 # point. This technique was first 
described by Gaddum (l933)» It has since been called the method of 
extreme effective doses. As he proposed it, however, there are drawbacks 
which severely limit its use. Most importantly for this discussion, there 
is no way given to describe the quality of the estimate. In addition, 
the estimate may be badly biased unless the levels used are symmetrically 
related to the response probabilities. These facts led Finney (1964) 
to advise that the method of extreme effective doses never be used. 

While rejecting the estimate, one should not ignore the importance 
of the low and high levels. As discussed above, there is some value in 
bypassing the estimation of the 50# point entirely and going directly to 
a treatment of quality, via confidence limits. It is in this approach 
that the low and high levels play a more appropriate role. In fact, for 
any integer c , we may take L to be c levels below the low level 
and U , c levels above the high level. (Fig. 1 provides an example 
of this procedure with c * 4.) 

There is one exception to this rule which is worth mentioning now. 
This is the case of perfectly consistent data. Here there is no overlap 
between the two responses, so that the high level occurs one step below 
the low level. In this situation, we take extra care and use the level 
c + 1 steps below the low level for L , as well as c + 1 steps above 
the high level for U . To put it another way, L should be the level 
c steps below the high level and U , c steps above the low level. 
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These aire the confidence limits as determined by the countback 
method. Now, what is it we're confident of as regards these limits'’ 

We are confident that, for intensities of level L and below, the 
probability that the subject will see a flash is less than . 5 ; while, 
for those of U and higher, this probability is greater than . 5 - Next, 
what proportion of the time will our confidence be misplaced? The degree 
of confidence associated with L and U cannot be specified exactly 
without introducing restrictive assumptions. With very modest pre- 
requisites, however, it can be proved that this value will always be 
greater than 1 - 2~ c . Thus, as noted in Figure 1, counting four 
2 _ 0 y© 1 s away fVom the low and high levels guarantees a confidence of 
more than 1 - 2~* = 15/l6 = . 9575 - Table 1 shows how this value 
increases with c . 

While the prerequisites for this result may be modest, it is 
important to make them explicit. The first requirement is that the 
observations be statistically independent. This is a standard assumption 
which allows us to write the probability of any set of responses as the 
product of the individual response probabilities. Intuitively, it means 
that response probabilities are not influenced by previously observed 
responses, and that no common influence acts on any group of responses. 

The second requirement for the countback method is that there be 
something like a unique 50# point. Specifically, for all levels of the 
variable above some point, one response (henceforth referred to as the 
positive response) should predominate. For all levels below, the other 
(negative) response should be more likely. It is not, of course, 
necessary for the experimenter to know where this point is, but merely 
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Table 1 

Lower Bound on Confidence for Countback 
Limits as a Function of c , the Number 
of Levels Counted Back 



Levels 

Counted Back 
c 


Lower Bound 
1 - 2~ C 


1 


.5000 


2 


.7500 


3 


.8750 


k 


.9375 


5 


.9688 


6 


.9844 


7 


.9922 


8 


.9961 


9 


.9980 


10 


.9990 
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that such a point exists. Intuitively, one may see this as requiring 
that the levels of the variable be ordered in a way which is mt.aningful 
for the response being studied and that the variable exert a significant 
influence on the response. 

Intuition has been made explicit for both these prerequisites 
because it is contemplated that they should be satisfied at an intuitive 
level. A satisfactory test of either assumption could probably not be 
made without considerably more data collection than the basic experi- 
ment would require. Thus, it is the responsibility of the experimenter 
who uses the countback method (or any of the others mentioned, for that 
matter) to design his experiment in such a way that these two assumptions 
may reasonably be made in his case. 

Taken together, the levels of the variable actually used in an 
experiment may be referred to as the experimental series. Once a 
potential user of the countback method has satisfied the two general 
prerequisites discussed above, he must still select an experimental 
series. The series should have two characteristics for satisfactory 
results using this technique. First, levels at the high end of the 
series should evoke a positive response almost invariably, while those 
at the low end should be similarly related to negative responses. 

Another way of saying this is that the experimental series should cover 
the range for which either response is a distinct possibility. 

Second, the experimental series should cover this range reasonably 
well. In particular, once a confidence level (and the corresponding c ) 
has been chosen, it would be foolish to use an experimental series whose 
length was less than 2c , and a length of 5c , as in Figure 1, is 



'ERIC 












r t 



?? SEE 



7 









- 11 - 



acceptable, but not at all extreme. (Here, length refers to the number 
of levels in a series.) 

Both of these recommendations are acceptable at an intuitive level. 
There is, however, a technical consideration which makes them important. 
Going back to Figure 1, suppose that a positive response has been 
obtained for intensity number 3. Then, with c = b , we would run out 
of levels while counting down to locate L . In such a situation, the 
theory stipulates that L be set at an artificial absolute lower limit 
for the variable, designated, for instance, by -« . The idea is that 
we have insufficient experimental information to locate L and we 
indicate this by writing L = -°° . At this value, L is considered 
to be a perfect (though useless) lower confidence limit. Similar 
remarks apply to U and the arbitrary upper limit, +» . 

This strange situation is a result of the fact that practically 
nothing has been assumed about the response probabilities at different 
levels. Consequently, only a reasonable amount of consistent data can 
establish nontrivial locations for L and U . The two recommendations 
made earlier simply put this observation in concrete terms. 

This is the countback method for constructing confidence limits. 

Now that we know how to use it, as well as the minimum to expect from it, 
the next step is to see how the countback method actually works. For 
this purpose, a few interesting examples have been selected and will be 
studied in some detail. 

Earlier it was noted that other approaches were not designed to 
handle data taking the form of one observation per level. Nonetheless, 
one of these methods, probit analysis, can be used with such data. More 
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gene rally, the principle underlying probit analysis, namely maximum 
likelihood, can be applied to the case of one observation per level! . 

As a consequence, some form of comparison between the countback method 
and traditional approaches is possible. This comparison is valuable for 
putting the countback method in perspective and is made in the forth- 
coming examples. 

To use maximum likelihood, a specific model relating response proba- 
bilities to levels of the variable must be assumed. A model which is 
often used (in probit analysis and the up-and-down method, for instance) 
is the Gaussian, or normal cumulative distribution. In the present case, 
the similar, but computationally simpler, logistic cumulative distri- 
bution was used instead. Formally, then, the probability of a positive 
response at level x was assumed to be (l + exp(-1.7a(x - b))) 1 . 

(This form was adapted from that used by Lord and Novick, 1968, p. 400.) 
Here a and b are parameters to be estimated (via maximum likelihood) 
from the data. One of the parameters, b , is the 50$ point and is of 
primary interest while the other, a , is related to the slope of the 
function when x = b and is used mainly to provide a better fit to the 
data. 

One of the primary virtues of maximum likelihood estimates is that 
they are asymptotically efficient and normally distributed. In practical 
terms, asymptotic efficiency means that no other estimates are better, 
given that the sample of data used is sufficiently large. 

Moreover, normally distributed estimates allow for a straightforward 
way of obtaining confidence limits for the parameter they are est ima ting. 
Upper and lower 95 $ limits, for example, are found by adding to and 
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subtracting from the estimate 1.96 times the standard deviation of the 
estimate, (in our case, we use the standard deviation of an efficient 
estimate, and this gives us, asymptotically, the best confidence limits 
for the parameter . ) 

These limits, of course, are only valid asymptotically and their 
actual confidence is not specified for small samples. This is an 
important part of the reason that maximum likelihood has not been con- 
sidered applicable to sensitivity data with one observation per level. 
Consequently, we should not be surprised by the behavior of these limits 
in the examples to be considered. 

In comparing maximum likelihood and the countback method, two 
artificial situations and one informal experiment were used. The experi- 
ment was one in which perception of depth in pictures was studied. It 
should be noted that this experiment was performed primarily to obtain 
realistic sample data. Many of the controls essential to a serious study 
of the problem were ignored and the results should be seen in that light . 

Subjects were shown pictures (such as the one in Pig. 2) of two 
rectangular panels at different angles to the picture plane and were 
asked to indicate which of the panels appeared wider. The design of the 
experiment provided that each subject would see four experimental series 
(randomly interspersed) and make one observation per level in each. A 
series was comprised of pictures showing a standard panel, with fixed 
width and slant, and a comparison panel with fixed slant and one of 20 
widths. Thus, each series had 20 levels. The 50$ point of such a series 
is the width at which the comparison appears as wide as the standard. 
Having four series allowed estimation of the 50$ points for four different 
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Fig. 2. Example of the pictures viewed by subjects in the experiment. 
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pairs of slants with each subject. There were 17 subjects, two of whom 
were used twice, so this experiment provided a total of 76 data sets of 
20 observations each on which to apply the two methods of analysis. 

Using maximum likelihood, nominal 95 $ and 99$ confidence limits 
were obtained for each data set. Using the countback method, limits 
were found for c * 1, 2, and 5* These limits are guaranteed to contain 
the 50$ point with probabilities greater than .5> .75 > and .875* respec- 
tively. 

One way of comparing these results is to look at the lengths of the 
confidence intervals obtained. These are given by U - L . To give some 
idea of the scale, it is worth noting that adjacent levels (widths measured 
on the plane of the picture) differed by .05 inches throughout all four 
series used. The median length of all 95$ confidence intervals provided 
by maximum likelihood is .1721 inches, comparable to the median length 
of intervals from the countback method with c = 1 , which is .15 inches. 
Moreover, 99$ maximum likelihood intervals have a median length of .2262 
inches, which compares with .25 inches for the countback method with 
c = 2 . The median length for c ■ 5 is .55 inches (see Table 2). 

In evaluating these comparisons , it should be realized that most of 
the data from this experiment were quite "good." That is, most of the 
data sets were either perfectly consistent or involved only a single 
reversal of positive and negative responses. Thus, our conclusions 
should be limited to this extent. For more complicated patterns of 
responses, maximum likelihood generally improves somewhat, relative to 
the countback method. For instance, consider the data in Figure 1. When 
c * 1 , L is 6 and U is 12. The 95$ limits are just above 6 and just 
below 11. For c = 2 , L becomes 5 and U , 15. The 99$ limits are 
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Table 2 

Median Lengths of the Confidence Intervals in 
the Experiment, as Obtained by Maximum 
Likelihood and the Countback Method 



Maximum Likelihood 


Countback Method 


95# . j.721 


c = 1 


.1500 


99# .2262 


c = 2 


.2500 




c = 3 


.3500 



me 
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just above 5 and just below 12. Thus, in this example, maximum likeli- 
hood intervals are shorter by about one step in each case. 

This improvement is particularly marked, of course, when counting 
back goes beyond the range of the experimental series. On the other 
hand, when maximum likelihood provides limits outside this range, strong 
assumptions are necessary to justify these limits. 

The data from this experiment also provide another possibility for 
comparing the two methods. When the experiment was designed, the 
hypothesis in mind was that most people see some depth in pictures, but 
not to the extent that the cues of linear perspective would predict. 
Thus, it was hoped that two hypotheses, which I shall call 2D and 3D, 
could each be rejected for most of the subjects. In terms of the data, 
these hypotheses predicted the location of 50 $ points for each of the 
four series used. The 3D 50$ points fell outside the range for all four 
series, while the 2D 50$ points were just outside (on the other side) 
for two, and well within for the other two. The ranges were chosen this 
way to provide an efficient use of the observations, on the assumption 
that most of the people would produce 50$ points quite different from 
those for either 2D or 3D. 

The hypotheses were meant to apply to the overall behavior of sub- 
jects, rather than to their responses on each of the series separately. 
Thus a criterion had to be chosen for deciding if either hypothesis 
applied to a given subject. The hypotheses predict 50$ points and the 
data give rise to confidence intervals. If at least three of the four 
predictions lay outside the corresponding intervals, the relevant 

hypothesis was rejected. This was felt to be fairly stringent, while 
allowing for some odd responses. 
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Applying this criterion to the data and using the four sets of 
confidence limits mentioned earlier gave the following results (pre- 
tending that there were 19 subjects, rather than 17 with two repeats). 
The 95# limits of maximum likelihood rejected both 2D and 3D for 17 
subjects, with one subject classified as a possible 2D and one as a 
possible 3D. Expanding the limits to 99# resulted in two more subjects 
classified as possibly 2D. 

The countback method, with c = 1 , gave results almost identical 
to those of 95# maximum likelihood. The only difference is that one 
additional subject was possibly 3D. Wlr^n c was set equal to 2, only 
13 subjects were neither 2D nor 3D, with three possibly 2D and three 
possibly 3D. The two 3D subjects not so classified by the 99# limits 
were actually the same subject. Consequently, one could say there is 
agreement to within one subject, both between 95# and c = 1 and 
between 99# and c = 2 , maximum likelihood being more powerful than 
the countback method in each case (see Table 3). 

We can now look at two artificial situations for added information 
about the two methods . The experiment allowed a study of the relative 
lengths of the confidence intervals, as well as of the relative ability 
to reject alternative hypotheses. An important question remains, how- 
ever, as to the actual confidence coefficients associated with the limits 
these two methods provide. (We have spoken of confidence coefficients 
before, but without giving them a name. The confidence coefficient for 
a set of limits is the probability that these limits contain the true 
value: .95, for example, for actual 93# limits.) The countback method, 

it will be remembered, states only an absolute lower bound for its 
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Table 3 

Results of Testing 2D and 3D Hypotheses with 
Maximum Likelihood (ML) and Countback (CB) 
Confidence Limits 



Method 


Number 

Possibly 

2D 


of Subjects 

Not 2D 
or 3D 


Possibly 

3D 


95# ML 


1 


17 


1 


CB, c - 1 


1 


16 


2 


99# ML 


3 


15 


1 


CB, c = 2 


3 


13 


3 


CB, c - 3 


3 


12 


4 
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confidence coefficients. Maximum likelihood, on the other hand, gives 
coefficients based on large samples drawn from the population speci- 
fied by its model. 

For purposes of analysis, the main thing that distinguishes 
artificial situations from actual experimental, ones is that, in the 
former, we choose the response probabilities and, thus, know them exactly. 
Consequently, when we apply a method of analysis to an artificial situa- 
tion, we know whether or not the assumptions of the method are being met. 

One way of studying an artificial situation is using it to generate 
artificial data. These data can be analyzed, confidence limits obtained, 
and the limits can be checked to see if they actually contain the 50$ 
point (since we know what the 50$ point is in an artificial situation). 
Repeating this procedure for many sets of data generated in the same way 
eventually provides a good estimate of how well the confidence limits 
are working. Specifically, the proportion of times the limits contain 
the 50$ point is an estimate of the confidence coefficient for these 
limits in this artificial situation. If the limits claim to have 95$ 
confidence, we hope the proportion would be close to .95* 

Although the above approach is quite straightforward, there are 
other ways of studying an artificial situation which are more satisfac- 
tory when they can be used. For discussion of these and related topics, 
the interested reader is referred to the Simulation Appendix. 

The first artificial situation was designed to see how well each 
method would fare when all the assumptions (except large sample size) 
of our version of the maximum likelihood approach were met. Consequently, 
the response probabilities for this situation were chosen according to 
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the ]Logistic model. In order to make the results of this analysis 
easily comparable with those of the experiment, the same 20 levels of 
the variable as appeared in two of the experimental series were used. 
Specifically, the values were .50, .55# .60, • •• # 1»^0, 1.45. For 
good measure, the 50 $ point was placed in the middle of this series, at 
.975 • The performance of the maximum likelihood and countback con- 
fidence limits (i.e., how often they contained .975) was then ascertained, 
as the steepness of the logistic curve varied. (Fig. 5a# b, c, and d 
shows curves with central slopes proportional to 2.5# 5# 10, and 20, 
respectively. They comprise most of the range over which the methods 
were tested.) 

In discussing the results, it is important to emphasize that the 
true confidence coefficients for both methods varied with the slope of 
the imderlying logistic curve. These coefficients tended to be lowest 
for a middle range of slopes, improving as the slopes got either very 
small or very large. 

The so-called 95$ confidence limits given by maximum likelihood had 
a confidence coefficient below .91 when the logistic curve had a slope 
of 10. The coefficient was still below .95 for slopes of 5 and 15# but 
rose to .967 when the slope increased to 20. The 99$ limits produced a 
coefficient less than .98 for a slope of 5# but were up well above .99 
for slopes of 15 or more. 

With c chosen as 1, the countback method guarantees only a con- 
fidence coefficient of .5» In this situation, however, it did consider- 
ably better, and was almost identical to 95$ maximum likelihood. When 
the slope was 8, its true coefficient reached a minimum of .879. Taking 
















t*i u " 






Response Probability Response Probability 



- 22 - 





1 




Level 

c 



Level 

d 



Pig. 3. Positive response probabilities for different levels as given by the logistic model, with slopes of 2.5, 5> 10, and 20. 
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c » 2 (a guaranteed confidence coefficient of .75 ) > the minimum co- 
efficient was .973* attained at a slope of 5* The coefficient 
increased to above .99 Tor slopes greater than 11. Finally, the 
coefficient for c = 3 never dropped below -99*+ (see the first part 
of Table 3) • 

It is worth summarizing and discussing these results. First, con- 
sider maximum likelihood. At worst, this approach underestimates the 
probability that its limits do not contain the 50 $ point by at least a 
factor of two. To see whether this difficulty was related to some 

special feature of maximum likelihood estimation, another method, namely 
2 

minimum X , was used. This method is not as widely used as maximum 

likelihood, but has been earnestly advocated (see Berkson, 19^9)* 

As with maximum likelihood, one must assume a specific probability 

2 

model to use minimum X , and, for consistency, the logistic was again 

2 

the model chosen. Minimum X is also guaranteed to produce asymptoti- 
cally efficient and normally distributed estimates as the sample size 
gets large, but here the relationship to maximum likelihood ends. It is 

an open question how well the two methods will agree for any specific 

2 

small sample situation. In fact, the minimum X results were virtually 
identical to those of maximum likelihood for the 95$ limits (giving a 
minimum true confidence coefficient of less than .91), and only slightly 
better for the 99$ limits (giving a minimum of less than . 985 ). 

Now let us consider the countback method. In contrast to maximum 
likelihood, it did much better than its claims. This fact led to the 
development of another theoretical result about countback confidence 
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limits . If response probabilities are symmetric 1 around the 50# point 
(as those of the logistic model are), the minimum confidence coefficient 
for the countback is 1 - 2' 2c (instead of 1 - 2” c ). Thus, for c = 1, 
symmetry guarantees a confidence of at least .75 and, for c = 2 , at 
least .9575 (see Table 4 for more values). 

This artificial situation has given us a better picture of the 
countback method than we had from the theory and experiment alone. In 
fact, there is more information to be gained. For each value of the 
slope, a (between 0 and 20 ), it was possible to find the median length 
of the countback confidence intervals. We can thus see how this length 
changes relative to & . When c = 5 (the only case studied) , the 
median interval is infinitely long when a is less than 3* For a 
between 3 and 8, the median length is approximately 3/a . Above 8, 
the median length is the minimum possible for c = 3 and a spacing 
between levels of .05 , namely .35* 

Evidently, the countback intervals are most efficient for a some- 
where between 3 and 8. For smaller values of a , the responses are not 
being obtained over a wide enough range of the variable of interest. For 
larger values, it seems that the range is more than adequate and a finer 
coverage (i.e., closer spacing between levels) is desirable instead. 

Now let us turn to a consideration of the second artificial situa- 
tion. The confidence limits of the countback method are at their worst 
when there is an asymmetry in the response probabilities. Such an 

^ere, symmetric means that the probability of a positive response k 
levels below the 50# point equals the probability of a negative response 
k levels above the 50# point, for k = 1,2,3,... . 
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Table k 

Lower Bound on Confidence Coefficient for 
Countback Limits When the Response 
Probabilities Are Symmetric 
Around the 50% Point 



Levels 

Counted Back 
c 


Lower Bound 
1 - 2 " 2c 


1 


.7500 


2 


.9375 


3 


. 98 L-I+ 


k 


.9961 


5 


.9990 
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asymmetry could involve a slow decrease in positive response probability 
below the 50 $ point and a rapid increase above this point. An extreme 
case of asymmetry occurs when the probability is .5 for all levels up 
to a point, above which the probability is 1. In this situation, the 
confidence coefficient for the countback method is just above 1-2 

In the second artificial situation the same levels were used as 
before, but the 50$ point was chosen to be between the sixth and seventh 
levels, at .79 on the scale used in the experiment. All levels above 
the sixth were assigned a positive response probability of 1. The 
remaining response probabilities were specified with the lower half of 
a cumulative logistic curve. The slope of this curve varied from 0 to 
20 on the same scale used before. The resulting situations, for slopes 
of 2.5 , 5> 10, and 20 , are shown in Figure ka., b, c, and d. When the 
slope is 0, we have the extreme case mentioned above, which results in 
a step function. 

In general, the confidence coefficients for all methods studied 
(countback, maximum likelihood, and minimum X ) increased as the slope 
of the lower half of the probability curve increased. Unlike the 
previous situation, the confidence coefficients in this case could be 
evaluated exactly for all methods. Consequently, the results can be 
easily summarized in graphical form. (See Fig. 5 on p. 50. Table 5 
also gives the results, along with those for the first situation.) 

There are several things to say in connection with these results. 
The countback with c = 1 differs from 95$ maximum likelihood only for 
small values of the slope. Maximum likelihood's 99$ limits are effec- 
tively identical to the 95$ limits attained by minimum X^ , hence the 
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Pig. 4. Positive response probabilities for different levels as given by an aaywetric Model, with lover ilqpte of 2.5, 5, 10, and 20* 
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Table 5 

Confidence Coefficients for the Countback Method (CB), Maximum 
Likelihood (ML), and Minimum X (MC) Confidence Limits in 
the Two Artificial Situations (Logistic and Asymmetric) 



Method 




Confidence 


Coefficients 








Logistic 

Slope 






Asymmetric 
Lower Slope 






5 


10 


15 


5 


10 


15 


CB, c = 1 


.90 


.89 


.93 


.73 


.90 


.967 


95# ML 


<•93 


<•91 


<.9*+ 


.73 


.90 


.965 


95# MC 


<•95 


<•91 


<.9^ 


.82 


.92 


.969 


CB, c - 2 


.973 


CO 

<0 

o\ 

• 


.997 


.94 


.991 


.999 


99# ML 


<•979 


<.988 


<•997 


.82 


.92 


.969 


99# MC 


<.986 


<.990 


<•998 


.958 


.993 


.999 


CB, c = 3 


.995 


>.999 


>.999 


.990 


>.999 


>.999 
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two labels for the same curve in Figure 5* The 99$ limits of minimum 
2 

X are better than those of the countback with c = 2 , but worse 
than with c = 3 . 

Based on the comparisons of the previous situations, the most 
surprising finding is the relative failure of the 99$ maximum likeli- 
hood confidence limits . This reinforces the observation made earlier 

2 

that maximum likelihood and minimum X need not give the same results 

2 

for small samples. In this case, the minimum X , with its generally 
longer confidence intervals, seems to have the advantage. For example, 
with a slope of 9 , the confidence coefficient for the 99$ minimum X^ 
limits is indeed .99, while the 99$ maximum likelihood limits have a 
coefficient of only .90. 

Finally, of all the limits studied, only those of the countback 
with c - 3 come close to being "safe” for an experimenter interested 
in confidence levels of at least .95* Again, it should be emphasized 
that the present example is an extreme case, and it may be considered 
likely that it represents the methods at something near their worst. 

Now we have seen how the countback method and two asymptotically 
efficient approaches (maximum likelihood and minimum X ) work in a few 
special cases. As was emphasized earlier, the latter two were designed 
to deal with data involving many observations per level and following a 
narrowly specified model of response probabilities. Consequently, the 
lack of correspondence between the confidence coefficients derived from 
these assumptions and those obtained in our cases should not be surpris- 
ing. This lack of correspondence should, however, strengthen our 
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reservations about naively applying maximum likelihood or minimum X 
to situations similar to those we have examined. 

The countback method gains additional support in these examples from 
the fact that its true confidence levels exceed the theoretical minimum 
values by a substantial margin. Thus it is comforting to know that one 
does not need to count back 5 steps in order to be reasonably sure of 
having 95# limits, or 7 steps for 99# limits. Instead, it appears that 
taking c = 2 for roughly symmetric situations, or c = 3 in more 
extreme cases will easily provide 95# limits, and increasing c by 1 
will increase the confidence to above 99#* 

Finally, from the point of view of design, using as few as 20 levels 
(hence 20 observations) seems to be satisfactory. Moreover, if there is 
some knowledge of the location of the 50# point, there is no need for 
extremely wide coverage with these levels. 

Thus a method starting with minimal assumptions about the data is 
able to operate effectively for an experimental design where traditional 
approaches are largely inappropriate. It is hoped that this development 
will encourage wider use of the sensitivity experiment with one observa- 
tion per level and its application to new research problems. 

We have studied the countback method only as it applies to data 
with one observation per level. There is, however, no reason why the 
same approach could not be used when more than one observation per level 
has been made. The high and low levels could still be the highest with 
a negative response and the lowest with a positive response, respectively. 
For an integer c , the lower limit L could still be c levels below 
the low level and the upper limit U , c levels above the high level. 
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The restrictions for perfect data and for artificial absolute limits 
could still be made. 

The only difference, in fact, would be in the lower bound for the 
confidence coefficient associated with L and U . Suppose, for 
simplicity, that the same number of observations, n , has been made at 
each level. Then the lower bound becomes 1 - 2 nc . Table 6 gives 
actual values of this bound for different choices of n and c . 
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Table 6 

Lower Bound on Confidence Coefficient 
for Countback Limits with More Than 
One Observation Per Level 



Levels 

Counted Back 
c 


n = 2 


Lower Bound 
1 - 2- nC 
n = 3 


n = 4 


1 


.7500 


.8750 


.9375 


2 


.9375 


.9844 


.9961 


3 


.9844 


.9980 


.9998 


4 


.9961 


.9998 


.99998 


5 


.9990 


•99997 


.999999 
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Statistical Appendix 



This appendix is divided into two sections. The first gives the 

essentials of the theory associated with the countback method. The 

2 

second section deals with the maximum likelihood and minimum X 
approaches as they apply to the logistic model and one observation 
per level. 

I * 

The Countback Method 

The main purpose of this section is to prove and expand upon asser- 
tions made earlier about the countback method of constructing confidence 
limits for a 50$ point in sensitivity experiments. It will be remembered 
that these limits were defined in terms of the lowest level at which a 
positive response was observed and the highest level at which a negative 
response was observed. Once a positive integer c is chosen, the lower 
limit, L , is defined as the level c steps below the low level. Simi- 
larly, the upper limit, U , is located c steps above the high level. 

If there are not enough levels in the experiment to allow L or U 
to be defined in this way, the artificial absolute limits, -°° or +°° , 
may be used for L or U , respectively. These are assumed to be 
absolute lower and upper limits . They are used to keep the confidence 

r 

associated with L and U within reasonable bounds, even when the data 
itself is unrewarding in at least one direction for the determination of 
a 50$ point. 

The other exception to the general countback rule occurs with perfect 
v data, i.e., a string of negative responses followed by a string of positive 
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responses. Here one counts back c + 1 steps rather than c . (Or, to 
put it another way, one counts down c steps from the high level for L 
and up c steps from the low level for U .) Briefly, the reason for 
this modification is that perfect data need not be as good as they look. 
Consequently, it is best to treat them with a little extra care. (This 
subject comes up again in the second section, where the problem is 
considerably more serious than at present.) 

There are two assumptions associated with the countback method. The 
first of these is that a level of the variable under study exists with 
the following property: the probability of a positive response at all 

higher levels is greater than .5 and less than .5 at all lower levels. 
(This level will occasionally be loosely referred to as the 50$ point.) 
The second assumption is that the observations are statistically inde- 
pendent . 

With these two prerequisites, it is possible to establish absolute 
lower bounds on the confidence associated with L and U , with these 
bounds depending only on c . 

In the discussion which follows, the experimental levels of the 
variable of interest will be designated 1,2, ...,N , from the lowest 
level to the highest. The probability of a positive response at level 
i will be abbreviated as P^ , with (= 1 - P^) referring to the 

probability of a negative response at level i . It will also be assumed 
that the 50 $ point falls well within the series, simply because the other 
cases are trivial extensions of this main one and add nothing new. 

Finally, the modification for perfect data will be ignored until 
later in the discussion. 
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If the 50 $ point lies between experimental levels i - 1 and i , 
then the assumption is that P. < .5 for j < i and P. > .5 for 

j > i . Now, in order for the limits L and U to be valid, L must 

be one of levels -°°,1,2,... , or i - 1 , and U must be one of levels 
i , i + 1, . . . ,N , or +°° . Put another way, L and U will not in- 
clude the 50$ point if L is any level greater than i - 1 , or if U 
is any level less than i . This latter point of view turns out to 
provide easier mathematics, so it is the one which has been adopted. 

L will be level i if the lowest positive response occurs at 
level i + c . More generally, L will be above level i - 1 if and 

only if the lowest level with a positive response is i + c or higher. 

This last is equivalent to saying that only negative responses occur at 

levels below i + c . The probability of such an event (by the inde- 

i+c-1 

pendence assumption) is n Q. . Similarly, U will be below 

3=1 J 

level i if and only if positive responses occur at all levels above 

N 

i - c - 1 . The probability of this event is II p. . 

j=i-c J 

The two events just described are mutually exclusive (i.e., it is 

impossible to have L above the 50$ point and U below it simultaneously) . 

As a consequence, the probability that L and U will, fail to include 

the 50$ point is just the sum of the individual probabilities: 
i+c-1 N 

II Q + II p. . Subtracting this quantity ffcom 1 gives the 
j=l J j—i— c J 

confidence coefficient associated with L and U . 

The next step is to find the minimum value this confidence coefficient 

can attain, given the constraints on the P. . This is equivalent to 

J 

maximizing the above sum. Clearly, the first term of the stun is greatest 
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when each of the Q. is maximized (j = 1, ... , i + c - l) . This 

J 

means setting Q. = 1 for j < i and Q. = .5 for j > i , according 

J t) 

Q 

to the constraints. The resulting maximum is .5 since there are c 

levels above i - 1 in the product. Incidentally, the second term 

is 0 under this choice of probabilities since setting Q. = 1 is 

J 

equivalent to setting P. = 0 . 

J 

In the same way, the second term can be maximized at .5 by taking 

maximal values for the P. , and this choice makes the first term 0 . 

J 

Thus the question remains as to whether the sum can attain a value 

greater than .5 for some compromise choice of the P. . In answering 

J 

this, it is important to note that the total probability of L and U 

failing is a linear function of each of the P. . This means that the 

J 

maximum value of this probability will be attained when each P. is 

J 

set at ones of its extremes . 

Now let us proceed by a process of elimination. Clearly, any choice 

of P. which makes either term 0 is unacceptable in trying to make the 
J 

Q 

sum exceed .5 • Consequently, all P. with j = i - c, ... , i + c - 1 

J 

must be set at .5* This is because all of these are represented in both 

terms, once as P. and once as 1 - P (= Q.) . For lower values of 

J J J 

j , only Q appears (so for these, P. may be set at 0 ). Similarly, 
J J 

for higher levels only P. appears, and P. may be taken as 1. Since 

J J 

there are 2c levels for which P. has been set at .5, the total 

J ’ 

<Os% O /-> Or* 1 

probability becomes: .5 + .5 = . 5 . This value is equal to 

.5° when c = 1 and less than .5° when c > 1 . 



The above proof is the justification for the claim that the count- 
back confidence coefficient is at least 1 - .5° when one counts back 
c steps . 
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Now we are ready to see what happens when the modification for 
perfect data is made. Before, a statement was made that L will he 
above i - 1 if and only if only negative responses occur below i + c . 
This is no longer true. If only negative responses occur below i + c 
and all higher responses are positive, L will be located at i - 1 . 
Consequently, the probability of this particular set of perfect data 
must be subtracted from our original value to describe the new situation. 
The result is 

i+c-1 i+c-1 N 

n Q. - ( it 0( n P ) . 

j=l 3 0=1 3 j=i+c 3 

In the same way, the probability of U being below the 50$ point is 
reduced to 

N i-c-1 N 

n p. - ( n « )( n p.) . 

j=i-c J j=l J j=i-c J 

Without going through any proof, one can see that this change can only 
increase the minimum confidence coefficient. The amount by which it is 
increased (namely .5^) is hardly worth mentioning, and 1 - .5° remains 
the practical lower bound. 

It is worth noting that the perfect data modification was made to 
help the method in typical rather than extreme cases. The minimum confi- 
dence coefficient, after all, is attained when there is perfect discrim- 
ination on one side of the 50$ point, and perfect confusion on the other 
side. This may be thought of as the maximally asymmetric case. Conse- 
quently, we may be led to ask what happens when the response probabilities 
are symmetric about the 50$ point. In one context, this may be taken to 
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mean that Q. - » P. , Q. _=P._, and so on. This, in turn, implies 
l-l 1 i-2 i+I 

that 

i+c-1 i+c-1 

n Q. = n P. , 

J-i-c J J=i-c J 

given that c+l<i<N-c+l. For ease of notation, let us set 

i-c-1 i+c-1 N 

II Q. = A , II Q. = B , and II P. = C 

j=l J J- i-c J j=i+c 3 

In these terms, the probability that L and U will fail to include 

the 50 $ point becomes AB + BC without the perfect data modification 

and AB - ABC + BC - ABC with it. These can be rewritten as B(A + C) 

and B(A + C - 2AC) , respectively. Because of the symmetry, we always 
2c 

have B < .5 . In addition, A + C can be as great as 2, while 

A + C - 2AC must always be less than 1. Consequently, when the 50$ 
point is well within the experimental series (c+l<i<N-c+l) , 
symmetry increases the lower bound for the confidence coefficient from 1 - .5° 

M *| p _ 

to 1 - .5 in the unmodified case and to 1 - .5 after the 

modification. These limits are listed in Table 7 . 

Finally, when the 50$ point is near either end of the series, 
symmetry becomes a less important constraint and the bound drops 
accordingly. In the extreme, when the 50$ point is outside the series, 
symmetry has no value and the bound returns to 1 - . 5 ° . 

2 

Maximum Likelihood and Minimum X Logistic Analysis 

These approaches differ fl-om the countback method in that they 




require a specific model of response probabilities. In the present work, 
the cumulative logistic distribution was taken as the model. If x is 
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Table 7 

Effect of Perfect-Data Modification on 



Lower Bound 


for Confidence 


Coefficient 


in 


the Symmetric 


Case 


Levels Counted Back 


Lower Bound 


c 


Without 

Modification 

1 - .5 20 - 1 


With 

Modification 
1 - . 5 2c 


1 


.5000 


•7500 


2 


.8750 


•9375 


3 


.9688 


.9844 


4 


.9922 


.9961 


5 


.9980 


.9990 




- 40 - 



the value of the variable at any level, it provides a value for the 
probability of a positive response at that level by 
P = (l + exp(-1.7a(x - b))) ^ . In this context, maximum likeli- 

r\ 

hood and minimum X^ are two ways of fitting this function to data 
by providing suitable choices for a and b . 

Let us first concentrate on maximum likelihood. Suppose we define 
r = 1 for a positive response at any level and r = 0 for a negative 
response. Then the probability of a given data set is HP r (l - p)^ r = L , 
where the product is taken over all experimental levels . Maximum 
likelihood estimation finds a and b which maximize L . Usually 
(and our case is no exception) this means solving the equations 
d log L/da = 0 and 3 log L/^b = 0 . (Taking logarithms does not 
change the location of the maximum and makes things mathematically 
simpler as well as numerically better behaved.) 

After some rearrangement, these equations become 
-1.7 Z(r - P)(x - b) = 0 and 1.7aZ(r - P) = 0 , which are equivalent 
to Zr = ZP and Zrx = ZPx . These equations were solved numerically 
with the help of a modified Newton- Raphs on method. The choice of 
starting values for a and b was important, but, given a good choice, 
a wide range of data could be fitted to the limits of computational 

accuracy, usually within six iterations. 

2 

For the minimum X method, a function other than the likelihood 
is important. As implied by the name, this approach finds a and b 
which minimize the usual measure of goodness of fit, namely 



= sQe ~ P) 2 + ((1 - r) - (1 - P)) 

L P (l - P) 



21 



= 2 If -■ P) 

pTT - p) 




In a manner analogous to maximum likelihood, this means solving 

2 2 

the equations d log X /da = 0 and d log X /db = 0 for a and b . 
Written out and simplified, these equations are 

(1.T/X 2 )[ E (x - b)e y - L (x - b)e~ y ] = 0 
r=0 r=l 

and 

(-1.7a/X 2 )[ L e y - Ze" y ] = 0 , 

r=0 r=l 

z \ 2 y -y 

where y = 1.7a(x - b) and X simplifies to £ e + 2 e . Again, 

r=0 r=l 

a modified Newton-Raphson procedure was applied, with much the same 
success as before. 

Having obtained estimates of the 50$ point (b in our logistic 

model), the next concern should be the quality of these estimates. 

2 

Both minimum X and maximum likelihood estimates can be shown to be 
asymptotically normally distributed around the true values with a 
variance- covariance matrix given by -D ^ (Kendall & Stuart, 1967). 

Here D is the expected value of the matrix of second-order derivatives 
of the log likelihood, evaluated at the true values of a and b . 

Thus, for instance, the element d p = E(d log L/dadb) . 

To pass from this information to nominal confidence limits requires 
quite an act of faith. Nonetheless, it is possible to evaluate the 
second-order derivatives at the estimated values of a and b and, 
from them, follow the theory to a matrix, S , analogous to the 
asymptotic variance-covariance matrix. Then s 22 is analogous to the 
variance of our estimate for b and, for instance, "95 $" confidence 
limits can be generated by subtracting and adding 1.96 Vs 00 to the 



estimate. It is, of course, a completely open question what the actual confi 

dence coefficient will be for these limits in any given situation. It 

was partly to study this question that the work described in the 

Simulation Appendix was carried out. 

Examples of computer output for selected data are found in Table 8. 

Here, as in the simulations and the experiment, there were 20 levels of 

the variable, from .50 to 1.1+5 in steps of .05. The output titled 

MAXILIK gives the results of applying maximum likelihood to the data, 

2 

while MINICHI gives the results of applying minimum X . Data appear 
following question marks, 0 for a negative response and 1 for a positive 
response. Below each data set are listed EST., the estimate, S.E., the 
standard error of estimate (Vs^ > based on the asymptotic results), and 
DER., the final derivative of the relevant function (log likelihood or 
log X ), for the slope (A) and 50 # point (B). Finally, " 95 #" and " 99 #" 
confidence intervals (C.I.) for the 50# point are given in parentheses. 

Comparing the results of the two programs for identical data 
suggests some generalizations. First and foremost, estimates for b , 
the 50# point, agree very closely. Second, minimum X^ provides a 
smaller estimate of the slope. (See Little, 1968, for a theoretical 
treatment of this point.) Third, maximum likelihood gives shorter con- 
fidence intervals. (Considering the simulation results, this should be 
seen as a failing rather than a virtue.) 

One more problem needs to be discussed. What happens to these 
methods with perfect data (i.e., data consisting of a string of negative 

responses, followed by a string of positive responses)? Both maximum 
2 

likelihood and minimum X find the best (indeed,perfect) fit to 



Table 8 



Examples of Output from Computer Programs Used to Find Maximum Likelihood 
(MAXILIK) and Minimum (MINICHl) Confidence Limit;’. 

MAXILIK 

? OyOfOfOfOfOfOyOfOylfOflflflflflflflflfl 



EST. S.E. 


DER. 


A 15.413 9.72732 


-1.77666 E-7 


B 0.975 4.36839 L-2 


-4.5755 E-8 



C.I. FOR B: C 0.88938 f 1.06062 ) C 0.86247 f 1.08753 ) 
7 0 v 0f0f0f0 9 l»0fl 9 lfl v l v l v l 9 l v l 9 l v l v l v l v l 



EST. S.E. 


DER. 


A 15.398 9.75583 


-1.60502 E-6 


B 0.774986 4.37133 E-2 


1.29816 E-6 



C.I. FOR B: ( 0.689308 v 0.860664 ) ( 0.662381 , 0.887591 ) 

? OfOfOfOfOfO'OflfOfOflflf 1 p 1 1 lflflflflfl 



EST. S.E. 


DER. 


A 10.7731 5.70744 


-1.53066 E-6 


B 0.924988 5.22584 E-2 


-1.7147 E-6 



C.I. FOR B: C 0.822562 f 1.02741 ) < 0.790371 # 1.05961 ) 

? OfO^OfOfOfOfOfOfOfOf 1 f 0 j 1 yOf 1 f 1 f 1 f 1 f 1 f 1 



EST. S.E. 


DER. 


A 8.72058 4.23208 


-6.28525 E-9 


B 1.07517 5.81569 E-2 


-4.24733 E-6 



C.I. FOR Bs C 0.961179 , 1.18915 ) C 0.925354 , 1.22498 ) 

? OfOfOfOfOflfl fOf 1 fOflfl fl flflflfl f 1 f 1 f 1 



EST. S.E. 


DER. 


A 6.55659 2.96189 


-6.9328 E-7 


B 0.823254 6.77866 E-2 


1.60414 E-6 



C.I. FOR B; C 0.690392 , 0.956116 ) C 0.648636 f 0.997872 ) 
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MINICHI 

? 0 9 0 9 0 9 0 9 0 9 0 9 0 9 0 9 0 9 1 9 0 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 
EST. S.E. DER. 

A 11.3192 6.13501 -6.04948 E-8 

B 0.975 5.09778 E-2 -1.16612 E-6 

C.I. FOR B! ( 0.875083 9 1.07492 ) C 0.843681 9 



? 0 9 0 9 0 9 0 9 0 9 1 9 0 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 
EST. S.E. DER. 

A 11.2691 6.22063 -3.25245 E-10 

B 0.774958 5.11698 E-2 -1.27135 E-6 

C.I. FOR B: C 0.674665 9 0.875251 ) C 0.643145 

? 0 9 0 9 0 9 0 9 0 9 0 9 0 9 1 9 0 9 0 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 
EST. S.E. DER. 

A 7.39583 3.3522 -2.95945 E-9 

B 0.913384 6.32176 E-2 -7.53317 E-7 

C.I. FOR Bi ( 0.789478 9 1.03729 ) C 0.750536 9 



7 0 9 0 9 0 9 0 9 0 9 0 9 0 9 0 9 0 9 0 9 1 9 0 9 1 9 0 9 1 9 1 9 1 9 1 9 1 9 1 
EST. S.E. DER. 

A 5.84153 2.51411 -2.72263 E-10 

B 1.07545 7.17866 E-2 -4.23514 E-7 

C.I. FOR 9: ( 0.934751 9 1.21615 ) ( 0.89053 9 

? 0 9 0 9 0 9 0 9 0 9 1 9 1 9 0 9 1 9 0 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 9 1 t 1 

EST. S.E. DER. 

A 4.32084 1.86446 -1.90648 E-9 

B 0.831057 8.67783 E-2 -1.40864 E-7 

C.I. FOR B: ( 0.660972 9 1.00114 ) ( 0.607517 9 



1.10632 ) 



0.906771 ) 



1.07623 ) 



1.26037 ) 



1.0546 ) 



perfect data when l/a = 0 . If l/a really were 0, only perfect data 
centered at the 50$ point would have any chance of occurring. Thus the 
variance of the estimate of the 50$ point would certainly he 0. This, 
however, leads us to the unacceptable conclusion that the length of the 
confidence intervals for perfect data should be 0: that we should have 

complete confidence in our estimate. Clearly, another approach must be 
taken. 

On intuitive grounds, one might want the confidence interval for 
perfect data to be at least as short as that for any other data which 
give the same estimate for b . In fact, the nonperfect data with the 
shortest associated confidence interval have a single reversal in the 
response pattern. Explicitly, single reversal data have a string of, 
negative responses, followed by a positive response, a negative, and a 
string of positives, in that order. 

Consequently, a conservative procedure in line with the intuition 
expressed above is to treat perfect data as though they were single 
reversal data, centered at the same place. For lack of any better idea, 
this is what has been done throughout the present study, both in the 
simulations and for the experimental data. It cannot be emphasized too 
strongly, however, that this is an unsolved problem, and an important 
one because of the relatively high likelihood of obtaining perfect data. 
(See Table 9 for the probability of perfect data with the probit model 
used in the simulation.) 
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Table 9 

The Probability of Obtaining Perfect Data 
with the Logistic Model Used in the First 
Simulation (20 Equally Spaced Levels, 
Centered on the 50$ Point) 



Slope of Logistic 
a 


Probability of 
Perfect Data 


2.5 


.0071 


5.0 


.0872 


7-5 


.2455 


10.0 


.4069 


12.5 


.5^05 


15.0 


.6443 


17.5 


.7236 


20.0 


.7842 



i 



o 

ERIC 



Simulation Appendix 



Before discussing the details of the simulations, it would be worth- 
while to describe the models used. For the standard situation, the 
logistic model provided values for P , the probability of a positive 
response at a given level of the variable: P = (l + exp(-1.7a(x - b))) ^ . 

In this equation, x is the value of the variable, b is the 50 $ point, 
and a is related to the slope of the logistic curve when x - b . 

(The constant factor 1.7 has been added to make a roughly comparable 
to l/cr in the normal distribution function.) 

To allow for comparisons with the accompanying experiment, twenty 
levels were used, with x varying fjrom .50 to 1.^5 in steps of .05 . In 
the standard situation, b was placed in the middle of this series, at 
•975 > and a was varied between 2.5 and 20. 

The extreme situation differed from this in only two respects. First, 
b was changed to .79* Second, all values of P corresponding to x > .79 
were taken to be 1, instead of the value given by the logistic. In other 
words, this situation operates on a logistic model below the 50 $ point, 
and on a deterministic (or errorless) one above it. 

Three kinds of simulation were applied to the above situations. In 
the first kind, sets of 20 independent values (each uniformly distributed 
between 0 and l) were obtained from a random number generator. If the 
i -th random number in a set was less than P for level i , it was 
taken to represent a positive response (and otherwise a negative one). 

In this way, sets of artificial data could be produced based on 
either situation and any choice of a . Moreover, these data could be 



analyzed by any of the three techniques to see whether or not b lay 
within the resulting confidence intervals. 

In fact, this approach was used only for the standard situation 
and then only with the maximum likelihood method. Five values of a 
were chosen (2.5, 5, 10, 15, and 20) and 1,000 sets of numbers were 
generated for each value. The performance of both "95$" and "99$ " 
confidence limits was observed for each of the samples. Finally, 99$ 
confidence limits based on the binomial distribution (Wilks, 1962 ) were 
calculated for each estimate of the confidence coefficients. (The 
results are given in Table 10.) 

The second kind of simulation involved a partial or complete 

enumeration of the possible response sets. In the extreme situation 

there are only 6k possible sets, since P = 1 for all but the six lowest 

levels. Thus, these 6k could each be analyzed, and those sets giving 

rise to intervals not including the 50$ point could be singled out. For 

any choice of a , the probability of obtaining each of these sets could 

be calculated directly. Thus, an exact value could be obtained for the 

confidence coefficient of limits generated by any of the three techniques. 

In fact, this approach was used only for maximum likelihood and minimum 

X . The results are shown in Figure 5 • (Note that the levels for "99$" 

2 

maximum likelihood limits are identical to those for "95 $" minimum X 
limits . This is a consequence of the fact that these two sets of limits 
failed to include the 50$ point for precisely the same sets of data.) 

Partial enumeration was used for the standard situation. The lowest 
five responses were fixed as negative and the highest five fixed as 
positive. All combinations of the middle 10 responses were then taken. 
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Table 10 

Estimates and 99$ Confidence Limits for Maximum 
Likelihood (ML) Confidence Coefficients, Based 
on Samples of 1000 Data Sets, in the 
Standard Situation (Logistic Model) 



Slope 






Confidence 


Coefficient 






a 




”95$" ML 






”99$" ML 






Lower 

Limit 


Esti- 

mate 


Upper 

Limit 


Lower 

Limit 


Esti- 

mate 


Upper 

Limit 


2.5 


.9249 


,9450 


.9619 


.9720 


.9840 


.9924 


5.0 


.8798 


.9050 


•9275 


.9618 


.9760 


.9367 


10.0 


.8742 


.9000 


.9231 


.9774 


.9880 


.9950 


15.0 


.9086 


.9310 


.9501 


.9844 


.9930 


.99796 


20.0 


.9520 


.9680 


.9806 


.9947 


.9990 


.999995 




Confidence Coefficient 
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2.'5 5.'o 7*5 io.'o 12!5 15-'o 17 .'5 2o!o 

Slope for Lower Half of Response Probability Curve 

Fig. 5* Confidence coefficients for the countback method (CB), 
maximum likelihood (ML), and minimum \ (MC) in the asymmetric situation, 
as a function of the lower slope. 
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and the resulting 1024 data sets analyzed, both by maximum likelihood 
2 

and minimum X . Again, the total probability of those sets whose 
intervals did not include the 50$ point was calculated for different 
values of a . 

Not all possible data were considered in this evaluation. The two 

extremes for the remainder are that none of the associated intervals 

include the 50$ point and that all of them do. Based on these extremes, 

2 

absolute bounds for the confidence coefficients were found. These 
bounds are quite good when a is large and almost useless when a is 
small (see Table ll). 

We can combine the results of Tables 10 and 11 to provide a com- 
posite picture of the maximum likelihood confidence coefficients for 
the standard situation. To do this, we simply take the better bounds 
on the confidence coefficients at each value of a . The results are 
given a natural representation (namely bands, within which the co- 
efficients almost certainly lie) in Figure 6, where they may be compared 
with the countback coefficients. 

The third kind of simulation is best described as analytic, and 
was used exclusively with the countback method. As shown in the 
* Statistical Appendix, exact formulas are available for the countback 

confidence coefficients in terms of P . These were used with several 

A 

values of a , and for c (the number of levels counted back) equal to 1, 2, 



The derivation of the absolute bounds is quite simple. Suppose 
p^ is the probability associated with the data examined whose intervals 
contain the 50$ point. Also suppose pg is the probability associated 
with the data not examined. Then the bounds on the true confidence 
coefficient are p^ and + Pg • 



o 
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Table 11 

Estimates and Absolute Bounds for Maximum Likelihood (ML) 

2 

and Minimum X (MC) Confidence Coefficients, Based on 
Partial Enumeration of the Possible Data Sets, 
in the Standard Situation (Logistic Model) 



Slope Method 
a 



Lower 

Bound 



2.5 


ML 


•1556 




MC 


.1591 


5.0 


ML 


.5546 




MC 


.5665 


7.5 


ML 


.7918 




MC 


• 7975 


10.0 


ML 


.8760 




MC 


.8776 


12.5 


ML 


.9152 




MC 


.9156 


15.0 


ML 


.9567 




MC 


•9568 


17.5 


ML 


.9540 




MC 


.9540 


20.0 


ML 


.9671 




MC 


.9671 



Confidence 
” 9 5 $" - • 



Esti- 

mate 


Upper 

Bound 


.9052 


.9860 


.9421 


• 9914 


.8927 


• 9333 


.9118 


.9452 


.8955 


.9056 


.9000 


.9114 


.9045 


.9075 


.9062 


.9091 


.9215 


.9220 


.9217 


.9224 


• 9590 


.9392 


• 9591 


• 9393 


• 95^7 


.95474 


• 95^7 


.95^76 


.9673 


.96731 


.9673 


.96731 



Coefficient 



Lower 

Bound 


"99#" 

Esti- 

mate 


.1423 


.9638 


.1445 


.9785 


.6000 


.9658 


.6068 


.9767 


.8656 


.9768 


.8708 


.9827 


.9562 


.9873 


.9586 


.9898 


.9852 


.9939 


.9861 


.99^8 


.99^8 


.99728 


• 9951 


•99757 


.9981 


.99884 


.9982 


.99892 


• 9993 


•99951 


• 9993 


•99954 



Upper 

Bound 

.9947 

•9968 

.9787 

.9855 

.9794 

.9846 

.9877 

.9901 

.99396 

. 99^85 

.99728 

.99758 

.99884 

.99892 

•99951 

•9995^ 



Confidence Coefficient 
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Fig. 6. Confidence coefficients for the countback method (CB) and bounds on 
the confidence coefficients for maximum likelihood (ML), based on Tables 10 and 11, 
for the logistic situation as a function of the slope. 
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and 3 in both the extreme and the standard situations. The results 
are summarized in Figures 5 and 6. 

Finally, median lengths of countback intervals were found through 
analytic simulation. The possible data for the standard situation were 
divided into equivalence sets, according to the length of the interval 
they would lead to. Due to the simplicity of the countback method, the 
probability for each set is expressible in an exact formula. 

To find the median interval length, one orders the sets according 
to length, begins with the smallest (for convenience) and accumulates 
the associated probabilities until the total reaches .5* The common 
length for the set of data last considered is the median of the lengths 
for all the data. The results of applying this procedure with c = 3 
are shown in Figure 7* Here a was varied between 2 and 20 but this 
has been reinterpreted as varying the spacing between levels with a 
fixed. Both spacing and lengths are given in units of l/a. 



o 
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Median Interval Length (Units of l/a ) 
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Fig. 7* Median length of eountback confidence intervals with c = 3 > for the 
logistic situation. 
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Historical Appendix 



Rather than provide a comprehensive history of approaches to the 
sensitivity problem, this section will attempt to give a general frame- 
work, so that the present development may be seen in proper perspective. 

In addition, more detailed discussions of various approaches will be 
referred to for the benefit of the interested reader. 

By far the most widely adopted method of dealing with sensitivity 
data is maximum likelihood estimation. (A small sample of recent 
applications is provided in a list immediately following the Bibliography.) 
Most appropriate in situations where a specific probability model arises 
from theoretical considerations, . it appeared in this context in a paper 
by Fisher (1922). In trying to estimate the density of organisms in a 
suspension, the dilution series experiment described at the beginning of 
the main section is usually employed. Fisher assumed that the number of 
organisms in any sample would have a Poisson distribution. (This would 
follow, as an adequately close approximation, from the assumption that 
the organisms are distributed randomly in the suspension.) On this basis, 
the probability that a given sample will have no organisms (resulting in 
a negative response) can be given explicitly in terms of the density of 
organisms in the suspension and the dilution of the sample. 

As a consequence, the probability of a given set of data can be 
explicitly stated. The value of the density which maximizes this proba- 
bility is the maximum likelihood estimate of the true density. Fisher 
worked out a way of obtaining this estimate and later provided tables 
(Fisher & Yates, 1963) to simplify the process. A good expository treatment 
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appears in a paper by Cochran (1950 ), and applications as well as statis- 
tical modifications continue to appear in large numbers . 

Unfortunately, most sensitivity problems do not come complete with 
a convenient and appropriate probability model. As a consequence, 
advocates of maximum likelihood estimation (e.g., Finney, 1952) have 
suggested the use of a standardized model, usually the cumulative normal 
distribution, with the provision that the variable under study be trans- 
formed, if necessary, to make the model more reasonable. 

To actually find the maximum likelihood estimates, the standard 
suggestion has been to convert the problem into a linear one and use 
weighted least squares. The conversion is made by transforming the 
observed proportion of positive responses at each level into its equiva- 
lent normal deviate (using the inverse of the cumulative normal function). 
As a convenience, the normal function usually used has a mean of 5 and a 
standard deviation of 1. In this case, the transformed proportions have 
been called probits (Bliss, 193 ^ )> and the name probit analysis has been 
applied to the technique as a whole. 

First correctly treated by Gaddum (1933) and Bliss (193 ^ ) > probit 
analysis has been discussed in detail by Finney (1952, 1964-), who includes 
a history of the method in his 1952 monograph. Another excellent review 
can be found in a collection of papers in Vol. 52 of the Annals of the 
New York Academy of Sciences . Of particular interest in this collection 
is a paper by Miller (1950 ) in which he discusses and compares varieties 
of probit analysis, related shortcut techniques, transformations other 
than the normal, and nonparametric methods . (These last will be mentioned 
later. ) 
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The applications of probit analysis are extremely widespread, mostly 
within biology and pharmacology but, to an increasing extent, in 
psychology as well. Statistical treatment and modification also con- 
tinue at a healthy pace. Much of this work can be found in Biometrics 
and some in Psychometrika . 

It may have been observed that probit analysis (as I described it) 
has an important failing which is highly relevant to the main subject 
of this paper. This is that the probit corresponding to the proportion 

0 is - 00 and, corresponding to 1, +°° . This is a problem, no matter 

how many responses are obtained at a level, but becomes acute when there 
is only one response per level, so that all proportions are either 0 or 
1. Both Bliss (1938) and Finney (19^7) have discussed this problem and 
offered reasonable solutions in terms of what Finney calls minimum and 
maximum working probits . Typically, this approach would involve several 
iterations of the weighted least squares solution. The alternative which 

1 adopted is to maximize the likelihood function directly (by numerical 
techniques) thus working only with the proportions. Needless to say, 
this alternative is facilitated by the existence of high-speed computers, 
a resource not available at the time probit analysis was developed. 

Another method using maximum likelihood and the normal model is the 
. "up-and-down" approach of Dixon and Mood (19^8). Here the main novelty 

is that the number of responses per level is determined as the experiment 
« progresses. This has the important advantage of concentrating observations 

near the 50 1 ° point, thus improving the precision of the estimate. It has 
the disadvantage that responses must be obtained on an individual basis, 
with the next level only chosen after the previous response is known. 
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This has limited its applications, particularly in some of the biological 
work discussed earlier. 

Hopefully, the foregoing remarks have provided some feeling for the 
extensive history maximum likelihood solutions to the sensitivity 
problem have enjoyed. To the nonstatistical reader, a word of explana- 
tion is in order. An extensive theory has been built up around maximum 
likelihood estimation. Among other things, this theory says that, 
asymptotically (i.e., as the sample of data becomes infinitely large), 
maximum likelihood estimates have minimum variance among all (asymptoti- 
cally) unbiased estimates . 

It is a fact, however, that maximum likelihood is not unique in 

2 

this regard. For instance, estimates obtained with minimum X 
techniques also have minimum variance as the sample becomes infinite. 

As the name implies, minimum X involves finding estimates which 
minimize the usual goodness -of -fit measure for the data with respect to 
some model (such as the normal cumulative distribution). 

Berkson is the main advocate of this approach for sensitivity data. 
He has discussed several variations of the method with both normt.1 and 
logistic models, and compared it with maximum likelihood, in numerous 
papers (19^6, 19b9, 1953* 1955a, 1955b, 1956). Applications of minimum 
X (as well as maximum likelihood) to psychological scaling problems 
are given a thorough treatment in a recent book by Bock and Jones (1968). 
Theoretical developments , as with maximum likelihood, are found mainly 
in Biometrics . 

Finally, there are several methods which are related to the count- 
back, in that no strict assumptions are made concerning the response 
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probabilities (although, unlike the countback, symmetry is always required). 
The best discussion of these approaches occurs in Finney (1^64 ), where 
extensive comparisons are made among them. A common difficulty with 

A . 

these methods is that they emphasize estimation of the 50/° point without 
providing much information about the quality of the results . 

The sole exception to this generalization is the Spearman-K&rber 
method (discussed by Finney, 1964, and Brown, I96l). It estimates the 
50$ point as a linear function of the number of positive responses at 
each level. Since the number of positive responses at any level is a 
binomial random variable, the variance of the estimate is a linear 
function of binomial variances. 

There has been no discussion on how to use this variance to con- 
struct confidence intervals for small samples, and the difficulty is 
acute for one observation per level, where one is not even sure how to 
estimate the variance itself. Nonetheless, Finney (1964) recommends the 
Spearman-Karber method for some cases of the sensitivity problem. 

The other method in this group which Finney recommends estimates 
the 50$ point using a moving average (of the number of positive responses 
per level) and linear interpolation. Approximate expressions for the 
variance of this estimate have been developed (Harris, 1959 ) > but none 
of these appears satisfactory for small samples. This moving average 
method is particularly unsuited for use with one observation per level. 

Not only is estimation of the variance problematical, there is also a 
real chance that its estimate of the 50$ point will not be uniquely 
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Experimental Appendix 



Introduction 

The conflict between the two- and three-dimensional aspects of a 
picture has long been recognized, particularly by painters. The 
experiment discussed in the main section was designed to study this 
conflict systematically. A door or window, when photographed from an 
angle, may give rise to an image in the form of a trapezoid whose bases 
are vertical lines . To reconstruct the width of the door or window 
from its image, one must consider several things . The two which will be 
considered here are the distance between the bases (the width of the 
image) and the angles between the sides and the bases (the linear per- 
spective). If we simplify the picture to include only this trapezoid, 
then its two- and three-dimensional aspects will be the two features 
just described. By obtaining judgments about the width of the original 
object as these features are varied, something may be learned about the 
nature of the conflict mentioned above. 



General Description 

Each subject was seated about two feet in front of a rear-projection 
screen. These were the instructions read to him: 

In this experiment, you will be shown a series of computer- 
generated pictures. Each of these shows two rectangular panels 
which meet at the center of the picture. You will be asked to 
indicate which of these panels appears wider to you by saying, 

"left" or "right." The panels are never of the same width, 
though they may be very similar. Also, no two pictures are 
identical, although some are very similar. 
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The first slide each subject saw is shown in Figure 2. The entire 
collection of 80 slides actually consisted of four subsets. These were 
constructed by choosing two standard panels, two comparison series, and 
combining each of the standards with each of the comparisons . The 
standards were both relatively flat in the picture plane and of moderate 
width, while the comparisons were more sharply angled toward the subject 
with widths varying systematically. (Thus, in Figure 2, the panel on the 
right is the standard, while that on the left is the comparison.) 

Since more or less independent judgments were desired, members of 
the same series were not shown in immediate succession and the side on 
which the standard panels appeared was varied. Once an order of 
presentation had been chosen, it was held constant for all subjects. 

In a more rigorous experiment, the order would have been varied randomly 
for different sub j ects . 

Equipment and Physical Specifications 

A Kodak Carousel projector with zoom lens and remotely controlled 
advance was used to show the slides. The rear-projection unit was manu- 
factured by the Radiant Corporation. It has a 14-inch square mylar 
screen, a sliding tray to hold the projector below and directly behind 
the screen, and two mirrors, one of them adjustable, to direct the picture 
from projector to screen. 

During the experiment, the room was completely dark except for a 
six-inch square lighted area in the center of the screen. The brightness 
of this area was brought to a comfortable viewing level by operating the 
projector at ’’low" and by using two pieces of Polaroid glass placed in 
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front of the projection lens as a variable filter. After this adjust- 
ment had been made, a Gossen footcandle meter was used to measure the 
illuminance of the screen at 25 inches, the approximate viewing distance 
of the subjects. The value arrived at in this way was 1.1 footcandles . 

From slide to slide, the length of the central vertical line (see 
Fig. 2) is kept constant. When the stimuli were produced, the ratio of 
this length to the viewing distance was taken to be .1. With the length 
of this vertical line at 2.5 inches in the projected image, the appro- 
priate viewing distance is 25 inches. 

In this kind of experiment, the usual procedure would be to provide 
a small viewing aperture 25 inches away from the screen and directly in 
front of it and to otherwise block the subject's forward view. This 
arrangement would have the additional advantage of restricting the sub- 
ject to monocular vision. In this way, the subject would have no 
binocular information about the flatness of the screen to contradict 
the impression of depth provided by the linear perspective. 

Nonetheless, we allowed the subjects a free binocular view of the 
stimuli. They were seated in chairs which put them at approximately 
the correct distance and orientation, but no attempt was made to restrict 
their movements or compensate for their varying heights. In part, this 
was due to the nonrigorous nature of the experiment. It was also due to 
the fact that this is the way we normally look at pictures. 

Production of the Stimuli 

As indicated in the instructions to the subjects, the slides used 
for this experiment were generated with the help of a computer, the 
IBM 709^ at Princeton University. It operates a cathode ray tube and a 
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35 mm. camera together so that images appearing on the former are photo- 
graphed on microfilm by the latter. The limits of image resolution in 
the finished product are fixed by the CRT rather than the film. All 
images must be built up on the face of the tube from a 1024 by 1024 square 
matrix of points. This is far short of ideal, but seemed satisfactory 
in the present case. 

The Geometry of Perspective 

To represent an object in perspective, the position of the object 
must be established with respect to a picture plane and a viewing point 
not in the plane. Imagine a line passing through the viewing point and 
any point in the object. The intersection of this line with the picture 
plane locates the representation of this part of the object. 

In terms of a three dimensional coordinate system, it is convenient 
to use the locus of points for which z - 0 (in other words, the x - y 
plane) as the picture plane, and to place the viewing point at (0, 0, p), 
along the z axis. If the object point is (x, y, z), its representation 
is (x’, y’, 0), where x* = (p/ (p - z))x and y* « (p/(p - z))y . This 
obviously excludes any part of the object for which z = p . 

The situation is represented in Figure 8. The values for x' and 
y* are found with the help of similar triangles: y' and y are the 

lengths of one set of corresponding sides, while x* and x are the 
lengths of another set. 

There are two opposing properties of perspective representation which 
we may note at this point. The first is that any change in the position 
of the object will, in general, result in a different representation. 
Moreover, this new representation will not, in general, even be 
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Fig. 8. The representation of point 0 as 0* when viewed from 
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geometrically similar to the old one. That is, the relationships of the 
parts of the representation will change as the object is moved. 

On the other hand, different objects may give rise to the same 
representation. In particular, a whole range of geometrically similar 
objects, increasing in overall size as they are placed further away from 
the viewing point, can be represented by a single figure on the picture 
plane . 

Rectangles in Perspective 

Having considered the basic projective equations, we should realize 
that they were not used directly to determine the stimuli for this 
experiment. Since the figures to be represented were rectangles, con- 
veniently aligned with the vertical axis of the picture plane, it was 
possible to derive special projective equations applying to them. 

Figure 9 provides a view of the situation from above (along the 
y -axis of Figure 8) as well as one from in front (along the z -axis). 
In the first of these views, BD is what can be seen of the rectangle 
and BC is what can be seen of its projection. The rectangle is in- 
clined from the picture plane by the angle 6 . Finally, A is the 
"vanishing point" of the rectangle, that is, the point toward which the 
top and bottom edges of its projection converge. 

In the experiment, the projections were varied systematically. As 
a result, the primary interest was in specifying the rectangles in terms 
of their projections (and the viewing distance). Referring to Figure 9, 
we want to find expressions for BD and Q in terms of AB , BC , and 
BP . 



o 








Fig. 9* The representation of a slanted rectangle when viewed from 
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Since triangles ACP and BCD are similar, BD/BC = AP/AC or 

BD = BC (AP/AC) . If we label the variables of interest as x = BD , 

a = AB , b = BC , and p = BP , the equation becomes 

2 2l/2 

x = (b/(a + b))(a + p ) ' . In addition, 6 = arctan (p/a) , so the 

problem is solved. 

Parameters of the Stimuli 

Now it is possible to describe the stimuli more completely. As we 
remarked earlier, the central vertical line (the common inside edge of 
the rectangles) was fixed, and its ratio to p , the projecting distance, 
was set at .1. Taking the length of the edge as 2.5 makes p = 25 . 

We may remember that in the experiment each of two standard stimuli 
is paired with each member of two comparison series. In the units set 
above (which happen to be inches), standard one (S.) has a = 50, 
b - 1.5 , while for , a = 25 , b = 1.5 • In other words, the 
standards have the same projected width. 

Let us use to denote the angle by which the top (or bottom) 
projected edge deviates from the horizontal. Then <t> for is 
arctan (1.25/50 ) = 1.1+3210° and, for S 2 , <t> = 2.86214-1° . 

Using the equations derived above, the rectangle represented by 
has a width of 1.62820 and comes out of the picture plane at an angle of 
26.5651° . That represented by S 2 has x = 2.0012k and 6 = 14-5° • 

We can summarize the above results in terms of an inverse projective 
function F : (b,<t>) -» (x,0) . 

For S 1 , F(1.5, 1.1+5210°) = (I.62820, 26.5651°) . 

For S 2 , F(1.5, 2.861+21°) = (2.00121+, 1+5.0000°) . 
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The two comparison series (C^ and C^) may be similarly described. 

For each of them, a (hence <t> ) is fixed while b is varied systemati- 
cally. In the case of , a = 2.5 (<t> = 26.5651°) and 

b = .50, .55* . • . ,1-45 . With C 2 , a = 5 (<*> = 14.0562°) and 

b = .75* .80, . . . ,1.70 . Thus each series has 20 components and the com- 

> ponents in each case are generated by changing b in steps of .05. 

We can easily describe the corresponding rectangles with the help 
of a modified version of F . 

For C 1 , F(. 50,.. .,1.45; 26.5651°) = (4.18745, ... ,9-22299; 84.2894°) . 
For C 2 , F( .75, . . . ,1.70; 14.0562°) = (5.52544, . ..,6.46890; 78.6901°) . 

Finally, the stimulus in Figure 2 is paired with an element of 

which has b « 1.15 • 

Data 

The data for all 19 subjects are found in Tables 12 and l4, 
immediately following this section. Table 12 gives the judgments in the 
order they were made, with an L if it was the left panel that appeared 
wider and an R if it was the right panel. Table 15 provides the code 
for interpreting these judgments. It may be read as follows: 1 Sl;Cl,l4 R 

means that the first judgment of each subject was made between and 

the l4th element of (ordered so that the first element is the 

smallest) with the standard on the right. 

Table l4 is the result of applying this code to Table 12. For ease 
(hopefully) of interpretation, the decoded data for each subject are 

3 

. Actually, there were only 17 subjects. Subject 7 wanted to try 

again, and, thus, become subject 8 as well. Subject 18 became subject 
19 using a different interpretation of the instructions. 
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Table 12 

Raw Data from Experiment with Each Subject's Judgment for 



Each of the 80 Slides, L and R Indicating Which 











Rectangle 


(Left 


or Right) Appeared Wider 










SUBJECT 


NUMBER 


1 








SUBJECT 


NUMBER 


2 








1 


L 


21 R 


41 


R 


61 


L 


1 L 


21 L 


41 


R 


61 


L 


2 


R 


22 L 


42 


L 


62 


R 


2 R 


22 R 


42 


L 


62 


R 


3 


R 


23 R 


43 


L 


63 


R 


3 R 


23 R 


43 


R 


63 


R 


4 


L 


24 R 


44 


R 


64 


L 


4 L 


24 R 


44 


R 


64 


R 


5 


L 


25 L 


45 


L 


65 


R 


5 L 


25 L 


45 


R 


65 


R 


6 


R 


26 L 


46 


L 


66 


R 


6 R 


26 L 


46 


L 


66 


R 


7 


L 


27 L 


47 


L 


67 


R 


7 L 


27 L 


47 


L 


67 


R 


8 


R 


28 R 


48 


L 


68 


L 


8 R 


28 R 


48 


L 


68 


L 


9 


R 


29 L 


49 


R 


69 


R 


9 R 


29 L 


49 


R 


69 


R 


10 


L 


30 L 


50 


L 


70 


R 


10 L 


30 L 


50 


L 


70 


R 


11 


R 


31 L 


51 


L 


71 


R 


11 L 


31 R 


51 


L 


71 


R 


12 


L 


32 R 


52 


L 


72 


L 


12 L 


32 L 


52 


L 


72 


R 


13 


L 


33 R 


53 


R 


73 


L 


13 L 


33 L 


S3 


L 


73 


L 


14 


R 


34 L 


54 


R 


74 


R 


14 R 


34 R 


54 


R 


74 


R 


IS 


R 


35 L 


55 


L 


75 


R 


15 R 


35 L 


55 


R 


75 


R 


16 


L 


36 L 


56 


L 


76 


R 


16 L 


36 L 


56 


L 


76 


R 


17 


L 


37 R 


57 


L 


77 


R 


17 R 


37 R 


57 


L 


77 


R 


18 


L 


38 L 


58 


L 


78 


L 


18 L 


38 L 


58 


R 


78 


L 


19 


R 


39 L 


59 


R 


79 


R 


19 R 


39 L 


59 


R 


79 


R 


20 


L 


40 R 


60 


R 


80 


R 


20 L 


40 R 


60 


R 


80 


R 



SUBJECT 


NUMBER 


3 








SUBJECT 


NUMBER 


4 








1 R 


21 L 


41 


L 


61 


L 


1 L 


21 L 


41 


R 


61 


L 


2 R 


22 R 


42 


L 


62 


R 


2 R 


22 L 


42 


L 


62 


R 


3 L 


23 L 


43 


R 


63 


L 


3 R 


23 L 


43 


R 


63 


R 


4 L 


24 R 


44 


L 


64 


R 


4 L 


24 R 


44 


L 


64 


R 


5 L 


25 R 


45 


R 


65 


L 


5 L 


25 L 


45 


L 


65 


R 


6 R 


26 L 


46 


L 


66 


R 


6 R 


26 L 


46 


L 


66 


R 


7 L 


27 R 


47 


R 


67 


L 


7 R 


27 L 


47 


L 


67 


R 


8 R 


28 R 


48 


L 


68 


L 


8 R 


28 R 


48 


L 


68 


L 


9 R 


29 L 


49 


L 


69 


L 


9 R 


29 L 


49 


R 


69 


R 


10 L 


30 L 


50 


R 


70 


R 


10 L 


30 L 


50 


L 


70 


R 


11 L 


31 R 


51 


L 


71 


L 


11 L 


31 R 


51 


L 


71 


R 


12 L 


32 L 


52 


L 


72 


R 


12 L 


32 L 


52 


L 


72 


R 


13 R 


33 L 


53 


L 


73 


R 


13 L 


33 L 


53 


L 


73 


L 


14 R 


34 R 


54 


L 


74 


R 


14 R 


34 R 


54 


R 


74 


R 


15 L 


35 L 


55 


R 


75 


R 


15 R 


35 L 


55 


R 


75 


R 


16 L 


36 L 


56 


L 


76 


R 


16 L 


36 L 


56 


L 


76 


R 


17 R 


37 R 


57 


R 


77 


R 


17 R 


37 R 


57 


L 


77 


R 


18 L 


38 L 


58 


R 


78 


L 


18 L 


38 L 


58 


R 


78 


L 


19 L 


39 L 


59 


R 


79 


L 


19 R 


39 L 


59 


R 


79 


R 


20 L 


40 R 


60 


L 


80 


R 


20 L 


40 R 


60 


R 


80 


R 
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Table 12 (Contd) 



SUBJECT 


NUMBER 


5 








1 


R 


21 L 


41 


R 


61 


L 


£ 


R 


22 R 


42 


L 


62 


R 


3 


L 


23 L 


43 


R 


63 


L 


4 


L 


24 L 


44 


L 


64 


R 


5 


L 


25 R 


45 


R 


65 


L 


6 


R 


26 L 


46 


L 


66 


R 


7 


L 


27 R 


47 


R 


67 


R 


8 


R 


28 R 


48 


L 


68 


L 


9 


R 


29 L 


49 


R 


69 


R 


10 


L 


30 L 


50 


L 


70 


R 


it 


L 


31 R 


51 


L 


71 


R 


12 


L 


32 L 


52 


L 


72 


R 


13 


R 


33 L 


53 


L 


73 


L 


14 


R 


34 L 


54 


R 


74 


R 


15 


L 


35 L 


55 


R 


75 


R 


16 


L 


36 L 


56 


L 


76 


R 


17 


R 


37 R 


57 


L 


77 


R 


18 


L 


38 L 


58 


R 


78 


L 


19 


L 


39 L 


59 


R 


79 


L 


£0 


L 


40 R 


60 


R 


80 


R 



SUBJECT NUMBER 7 



1 


L 


21 R 


41 


R 


61 


L 


2 


R 


22 L 


42 


L 


62 


R 


3 


R 


23 R 


43 


L 


63 


R 


4 


L 


24 R 


44 


R 


64 


L 


5 


R 


25 L 


45 


L 


65 


R 


6 


R 


26 R 


46 


L 


66 


R 


7 


R 


27 L 


47 


L 


67 


R 


8 


R 


28 R 


48 


L 


68 


R 


9 


R 


29 L 


49 


R 


69 


R 


10 


L 


30 L 


50 


L 


70 


R 


11 


R 


31 L 


51 


L 


71 


R 


12 


L 


32 R 


52 


L 


72 


R 


13 


L 


33 R 


53 


R 


73 


L 


14 


R 


34 L 


54 


R 


74 


R 


IS 


R 


35 R 


55 


L 


75 


L 


16 


L 


36 L 


56 


R 


76 


L 


17 


R 


37 R 


57 


L 


77 


L 


18 


R 


38 L 


58 


L 


78 


L 


19 


R 


39 R 


59 


R 


79 


R 


20 


L 


40 R 


60 


R 


80 


R 



SUBJECT 


NUMBER 


6 








1 


L 


21 L 


41 


R 


61 


L 


2 


R 


22 L 


42 


L 


62 


R 


3 


R 


23 R 


43 


L 


63 


R 


4 


L 


24 R 


44 


R 


64 


L 


5 


L 


25 L 


45 


L 


65 


R 


6 


R 


26 L 


46 


L 


66 


R 


7 


L 


27 L 


47 


L 


67 


R 


8 


R 


28 R 


4B 


L 


68 


L 


9 


R 


29 L 


49 


R 


69 


R 


10 


L 


30 L 


50 


L 


70 


R 


11 


R 


31 L 


51 


R 


71 


R 


12 


L 


32 L 


52 


L 


72 


R 


13 


L 


33 R 


53 


L 


73 


L 


14 


R 


34 L 


54 


R 


74 


R 


15 


R 


35 L 


55 


L 


75 


L 


16 


L 


36 L 


56 


L 


76 


R 


17 


R 


37 R 


57 


L 


77 


R 


18 


L 


38 L 


58 


L 


78 


L 


19 


R 


39 R 


59 


L 


79 


R 


20 


L 


40 R 


60 


R 


80 


R 



SUBJECT 


NUMBER 


8 








1 


L 


21 R 


41 


R 


61 


L 


2 


R 


22 L 


42 


L 


62 


R 


3 


R 


23 R 


43 


R 


63 


R 


4 


L 


24 R 


44 


R 


64 


R 


5 


R 


25 L 


45 


R 


65 


R 


6 


R 


26 R 


46 


L 


66 


R 


7 


L 


27 L 


47 


L 


67 


R 


8 


R 


28 R 


48 


L 


68 


R 


9 


R 


29 L 


49 


R 


69 


R 


10 


L 


30 R 


50 


L 


70 


R 


11 


R 


31 R 


51 


L 


71 


R 


12 


L 


32 R 


52 


L 


72 


R 


13 


L 


33 R 


S3 


R 


73 


L 


14 


R 


34 L 


54 


R 


74 


R 


15 


R 


35 R 


55 


L 


75 


L 


16 


L 


36 L 


56 


L 


76 


R 


17 


R 


37 R 


57 


L 


77 


R 


18 


R 


38 R 


58 


L 


78 


L 


19 


R 


39 R 


59 


L 


79 


R 


20 


L 


40 R 


60 


R 


80 


R 
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Table 12 (Contd) 



SUBJECT NUMBER 


9 






SUBJECT 


NUMBER 


10 








1 R 


21 L 


41 


R 


61 L 


1 L 


21 R 


41 


R 


61 


L 


2 R 


22 R 


42 


L 


62 R 


2 R 


22 L 


42 


L 


62 


R 


3 R 


23 L 


43 


R 


63 L 


3 R 


23 R 


43 


L 


63 


R 


4 L 


24 R 


44 


L 


64 R 


4 L 


24 R 


44 


R 


64 


L 


5 L 


25 R 


45 


R 


65 L 


5 R 


25 L 


45 


L 


65 


R 


6 R 


26 L 


46 


L 


66 R 


6 R 


26 R 


46 


L 


66 


R 


7 L 


27 R 


47 


R 


67 R 


7 R 


27 L 


47 


L 


67 


R 


8 R 


28 R 


48 


L 


68 L 


8 R 


28 R 


48 


L 


68 


L 


9 R 


29 L 


49 


R 


69 R 


9 R 


29 L 


49 


R 


69 


R 


10 L 


30 L 


50 


L 


70 R 


10 L 


30 R 


50 


L 


70 


R 


11 L 


31 R 


51 


L 


71 R 


11 R 


31 L 


51 


R 


71 


R 


12 L 


32 L 


52 


L 


72 R 


12 L 


32 R 


52 


L 


72 


L 


13 R 


33 L 


53 


L 


73 L 


13 L 


33 R 


S3 


R 


73 


L 


14 R 


34 R 


54 


R 


74 R 


14 R 


34 L 


54 


R 


74 


R 


is R 


35 L 


55 


R 


75 R 


15 R 


35 R 


55 


L 


75 


L 


16 L 


36 L 


56 


L 


76 R 


16 L 


36 L 


56 


R 


76 


L 


17 R 


37 R 


57 


L 


77 R 


17 R 


37 R 


57 


L 


77 


R 


18 L 


38 L 


58 


R 


78 L 


18 R 


38 R 


58 


L 


78 


R 


19 L 


39 L 


59 


R 


79 L 


19 R 


39 R 


59 


L 


79 


R 


^0 L 


40 R 


60 


L 


80 R 


20 L 


40 L 


60 


R 


80 


L 



SUBJECT NUMBER 


11 




• 




SUBJECT 


NUMBER 


12 








1 L 


21 L 


41 


R 


61 


L 


1 R 


21 L 


41 


R 


61 


L 


2 R 


22 R 


42 


L 


62 


R 


2 R 


22 R 


42 


L 


62 


R 


3 R 


23 R 


43 


R 


63 


R 


3 R 


23 L 


43 


R 


63 


R 


4 L 


24 R 


44 


L 


64 


R 


4 L 


24 R 


44 


R 


64 


R 


5 L 


25 L 


45 


R 


65 


R 


5 L 


25 R 


45 


R 


6S 


L 


6 R 


26 L 


46 


L 


66 


R 


6 R 


26 L 


46 


L 


66 


R 


7 L 


27 L 


47 


R 


67 


R 


7 L 


27 R 


47 


R 


67 


R 


8 R 


28 R 


48 


L 


68 


L 


8 R 


28 R 


48 


L 


68 


L 


9 R 


29 L 


49 


R 


69 


R 


9 R 


29 L 


49 


R 


69 


R 


10 L 


30 L 


50 


L 


70 


R 


10 L 


30 L 


50 


L 


70 


R 


11 L 


31 R 


51 


L 


71 


R 


11 L 


31 R 


51 


L 


71 


R 


12 L 


32 L 


52 


L 


72 


R 


12 L 


32 L 


52 


L 


72 


R 


13 L 


33 L 


53 


L 


73 


L 


13 R 


33 L 


53 


L 


73 


L 


14 R 


34 L 


54 


R 


74 


R 


14 R 


34 R 


54 


R 


74 


R 


15 R 


35 L 


55 


R 


75 


R 


15 R 


35 L 


55 


R 


75 


R 


16 L 


36 L 


56 


L 


76 


R 


16 L 


36 L 


56 


L 


76 


R 


17 R 


37 R 


57 


L 


77 


R 


17 R 


37 R 


57 


L 


77 


R 


18 L 


38 L 


58 


L 


78 


L 


18 L 


38 L 


58 


R 


78 


L 


19 R 


39 L 


59 


R 


79 


R 


19 L 


39 L 


59 


R 


79 


R 


20 L 


40 R 


60 


R 


80 


R 


20 L 


40 R 


60 


L 


80 


R 
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Table 12 (Contd) 



SUBJECT 


NUMBER 


13 








SUBJECT NUMBER 


14 








1 L 


21 R 


41 


R 


61 


L 


1 L 


21 L 


41 


R 


61 


L 


2 R 


22 L 


42 


L 


62 


R 


2 R 


22 R 


42 


L 


62 


R 


3 R 


23 R 


43 


L 


63 


R 


3 R 


23 L 


43 


R 


63 


L 


4 L 


24 R 


44 


R 


64 


L 


4 L 


24 R 


44 


L 


64 


L 


5 L 


25 L 


45 


L 


65 


R 


5 L 


25 L 


45 


L 


65 


R 


6 R 


26 L 


46 


L 


66 


R 


6 R 


26 L 


46 


L 


66 


R 


7 L 


27 L 


47 


L 


67 


R 


7 L 


27 R 


47 


L 


67 


R 


8 R 


28 R 


48 


L 


68 


R 


8 R 


28 R 


48 


L 


68 


L 


9 R 


29 L 


49 


R 


69 


R 


9 R 


29 L 


49 


R 


69 


R 


10 L 


30 L 


50 


L 


70 


R 


10 L 


30 L 


50 


L 


70 


R 


11 R 


31 L 


51 


R 


71 


R 


11 L 


31 R 


51 


L 


71 


R 


12 L 


32 L 


52 


L 


72 


L 


12 L 


32 L 


52 


L 


72 


R 


13 L 


33 L 


53 


R 


73 


L 


13 L 


33 L 


53 


L 


73 


L 


14 R 


34 L 


54 


R 


74 


R 


14 R 


34 L 


54 


R 


74 


R 


15 R 


35 L 


55 


L 


75 


L 


15 R 


35 R 


55 


L 


75 


L 


16 L 


36 L 


56 


R 


76 


L 


16 L 


36 L 


56 


L 


76 


R 


17 L 


37 R 


57 


L 


77 


R 


17 R 


37 R 


57 


L 


77 


R 


18 R 


38 L 


58 


L 


78 


R 


18 L 


38 L 


58 


L 


78 


L 


19 R 


39 R 


59 


L 


79 


R 


19 R 


39 L 


59 


R 


79 


R 


20 L 


40 L 


60 


R 


80 


L 


20 L 


40 R 


60 


R 


80 


L 



SUBJECT 


NUMBER 


15 








SUBJECT 


NUMBER 


16 








1 L 


21 L 


41 


R 


61 


L 


1 L 


21 L 


41 


R 


61 


L 


2 R 


22 L 


42 


L 


62 


R 


2 R 


22 L 


42 


L 


62 


R 


3 R 


23 R 


43 


R 


63 


R 


3 R 


23 R 


43 


L 


63 


R 


4 L 


24 R 


44 


R 


64 


L 


4 L 


24 R 


44 


R 


64 


L 


5 L 


25 L 


45 


R 


65 


R 


5 L 


25 L 


45 


L 


65 


R 


6 R 


26 L 


46 


L 


66 


R 


6 R 


26 L 


46 


L 


66 


R 


7 L 


27 L 


47 


L 


67 


R 


7 L 


27 L 


47 


L 


67 


R 


8 R 


28 R 


48 


L 


68 


L 


8 R 


28 R 


48 


L 


68 


L 


9 R 


29 L 


49 


R 


69 


R 


9 R 


29 L 


49 


R 


69 


R 


10 L 


30 L 


50 


L 


70 


R 


10 L 


30 L 


50 


L 


70 


R 


11 R 


31 R 


51 


L 


71 


R 


11 L 


31 L 


51 


L 


71 


R 


12 L 


32 R 


52 


L 


72 


R 


12 L 


32 L 


52 


L 


72 


L 


13 L 


33 L 


53 


L 


73 


L 


13 L 


33 L 


53 


L 


73 


L 


14 R 


34 L 


54 


R 


74 


R 


14 R 


34 L 


54 


R 


74 


R 


15 R 


35 L 


55 


L 


75 


L 


IS R 


35 L 


55 


L 


75 


L 


16 L 


36 L 


56 


L 


76 


L 


16 L 


36 L 


56 


L 


76 


L 


17 R 


37 R 


57 


L 


77 


R 


17 R 


37 R 


57 


L 


77 


R 


18 L 


38 L 


58 


L 


78 


L 


18 L 


38 L 


58 


L 


78 


L 


19 R 


39 L 


59 


R 


79 


R 


19 R 


39 R 


59 


R 


79 


R 


20 L 


40 R 


60 


R 


80 


L 


20 L 


40 R 


60 


R 


80 


L 
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Table 12 (Contd) 



SUBJECT 


NUMBER 


17 








1 


R 


21 L 


41 


L 


61 


R 


£ 


L 


22 R 


42 


L 


62 


R 


3 


L 


23 L 


43 


R 


63 


L 


4 


L 


24 R 


44 


L 


64 


R 


5 


L 


25 R 


45 


R 


65 


L 


6 


R 


26 L 


46 


L 


66 


R 


7 


L 


27 R 


47 


R 


67 


R 


8 


R 


28 R 


48 


L 


68 


L 


9 


R 


29 L 


49 


R 


69 


R 


10 


L 


30 L 


50 


R 


70 


R 


11 


L 


31 R 


51 


L 


71 


L 


12 


L 


32 L 


52 


L 


72 


R 


13 


R 


33 L 


53 


L 


73 


R 


14 


R 


34 R 


54 


L 


74 


R 


15 


L 


35 L 


55 


R 


75 


R 


16 


L 


36 L 


56 


L 


76 


R 


17 


R 


37 R 


57 


R 


77 


R 


18 


L 


38 L 


58 


R 


78 


L 


19 


L 


39 L 


59 


R 


79 


L 


20 


L 


40 R 


60 


L 


80 


R 



SUBJECT NUMBER 18 



1 


R 


21 


L 


41 


L 


61 


L 


2 


L 


22 


R 


42 


L 


62 


R 


3 


L 


23 


L 


43 


R 


63 


L 


4 


L 


24 


L 


44 


L 


64 


R 


5 


L 


25 


R 


45 


R 


65 


L 


6 


R 


26 


L 


46 


L 


66 


R 


7 


L 


27 


R 


47 


R 


67 


L 


8 


R 


28 


R 


48 


L 


68 


L 


9 


L 


29 


R 


49 


L 


69 


L 


10 


R 


30 


L 


50 


R 


70 


R 


1 1 


L 


31 


R 


51 


L 


71 


L 


12 


L 


32 


L 


52 


L 


72 


R 


13 


R 


33 


L 


53 


L 


73 


R 


14 


R 


34 


R 


54 


L 


74 


L 


15 


L 


35 


L 


55 


R 


75 


R 


16 


L 


36 


L 


56 


L 


76 


R 


17 


R 


37 


R 


57 


R 


77 


R 


18 


L 


38 


L 


58 


R 


78 


L 


19 


L 


39 


L 


.59 


R 


79 


L 


20 


L 


40 


R 


60 


L 


80 


R 



SUBJECT 


NUMBER 


19 






1 L 


21 L 


41 R 


61 


L 


2 R 


22 R 


42 L 


62 


R 


3 R 


23 L 


43 R 


63 


R 


4 L 


24 R 


44 L 


64 


R 


5 L 


25 L 


45 R 


65 


R 


6 R 


26 L 


46 L 


66 


R 


7 L 


27 L 


47 L 


67 


R 


8 R 


28 R 


48 L 


68 


L 


9 R 


29 L 


49 R 


69 


R 


10 L 


30 L 


50 L 


70 


R 


11 L 


31 R 


51 L 


71 


R 


12 L 


32 L 


52 L 


72 


R 


13 L 


33 L 


53 L 


73 


L 


14 R 


34 R 


54 R 


74 


R 


15 R 


35 L 


55 L 


75 


R 


16 L 


36 R 


56 L 


76 


R 


17 R 


37 R 


57 L 


77 


R 


18 L 


38 L 


58 L 


78 


L 


19 R 


39 L 


59 R 


79 


R 


20 L 


40 R 


60 R 


80 


R 
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'jOKaoXTE 



- 75- 

Table 13 

Arrangement of Slides for Experiment, Showing Which Standard 
Rectangle Was Paired with Which Comparison, and 
on Which Side the Standard Appeared 



1 


Si 


C 1 * 1 4 


R 


21 


SI 


Cl, 


5 


L 


41 


S2 


Cl, 


14 


L 


61 


S2 


Cl, 


19 


R 


2 


S2 


C2, 13 


L 


22 


S2 


C2, 


11 


R 


42 


S2 


C2, 


19 


R 


62 


SI 


C2, 


19 


L 


3 


S2 


C 1 , 1 6 


L 


23 


SI 


Cl, 


9 


L 


43 


SI 


Cl, 


4 


R 


63 


S2 


Cl, 


11 


L 


4 


SI 


C2, 1 / 


R 


24 


SI 


C2, 


12 


L 


44 


S2 


C2 , 


10 


L 


64 


SI 


C2, 


8 


R 


5 


S2 


Cl , 1 


L 


25 


SI 


Cl, 


13 


R 


45 


SI 


Cl, 


6 


R 


65 


SI 


Cl, 


10 


L 


6 


SI 


C2,3 


R 


26 


SI 


C2, 


2 


L 


46 


S2 


9k 

CO 

o 


18 


R 


66 


S2 


C2 , 


,3 


R 


1 


SI 


Cl, 3 


L 


27 


S2 


Cl, 


13 


R 


47 


S2 


Cl, 


10 


R 


67 


S2 


Cl, 


,20 


L 


S 


St 


o 

to 


L 


28 


S2 


C2, 


1 


R 


4b 


S2 


C2, 


,2 


L 


68 


S2 


C2| 


,8 


L 


9 


SI 


C 1 , 1 H 


L 


29 


S2 


Cl, 


18 


R 


49 


SI 


Cl, 


,17 


L 


69 


SI 


Cl, 


,19 


L 


n 


SI 


C2, 1 4 


R 


30 


S2 


C2 , 


6 


L 


50 


SI 


C2, 


,13 


R 


70 


S2 


C2, 




R 


n 


S2 


C 1 ,6 


L 


31 


S2 


Cl, 


‘■) 


R 


51 


SI 


Cl, 


,1 


L 


71 


S2 


Cl, 


,12 


L 


12 


S2 


C2 , 1 6 


R 


32 


SI 


C2, 


7 


L 


52 


S2 


C2 , 


,20 


R 


72 


SI 


C2, 


,6 


R 


13 


S2 


C 1 , 1 5 


R 


33 


S2 


Cl, 


5 


L 


53 


S2 


Cl, 


, 4 


L 


73 


SI 


Cl, 


,15 


R 


1 4 


SI 


C2, 1 5 


L 


34 


SI 


C2 , 


1 1 


R 


54 


S2 


C2, 


,14 


L 


74 


SI 


C2, 


,is 


L 


1 7 


Si 


C 1 , 1 6 


L 


35 


SI 


Cl, 


2 


L 


55 


SI 


Cl, 


,7 


R 


75 


SI 


Cl, 


, 8 


R 


16 


Si 


C2,20 


R 


36 


S2 


C* f 


17 


R 


56 


SI 


C2, 


,5 


L 


76 


S2 


C2, 


,9 


R 


17 


S2 


Cl, 3 


R 


37 


SI 


Cl, 20 


L 


57 


S2 


Cl 


,17 


R 


77 


S2 


Cl, 


,2 


R 


18 


S2 


o 

ro 

m 

•*4 


L 


38 


SI 


C2, 


,1 


L 


58 


SI 


C2 


,10 


R 


78 


S2 


C2, 


,4 


L 


19 


SI 


Cl , 12 


L 


39 


S2 


Cl, 


,7 


L 


59 


S2 


Cl 


,8 


R 


79 


SI 


Cl 


,11 


L 


20 


S2 


C2, 1 5 


R 


40 


SI 


C2, 


,4 


h 


60 


S2 


C2 


,12 


L 


SO 


SI 


C2 


,9 


R 



=ar*z: 
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Table 14 

The Data of Table 12, Rearranged Using Table 13 So That for 
Each Comparison Width, It Is Noted Whether the Standard 
(-1) or the Comparison (l) Was Judged Wider 



SUBJECT NUMBER 1 



SUBJECT NUMBER 2 







Cl 


C2 








Cl 


C2 




B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


SI 


S2 


0.30 


_ | 


. | 






0.50 


• | 


• | 






0.55 


- 1 


m I 






0.55 


- 1 


- 1 






o.so 


- 1 


1 






0.30 


- l 


- 1 






0.55 


1 


1 






0.35 


- l 


• 1 






0.70 


1 


1 






0.70 


- 1 


- 1 






0.75 


1 


1 


• 1 


-1 


0.75 


- 1 


- 1 


- 1 


- I 


0.80 


1 


m I 


- 1 


-1 


0.30 


- l 


- 1 


m 1 


- 1 


0.85 


- 1 


- 1 


m I 


-1 


0.85 


- 1 


- 1 


- I 


- I 


0.90 


1 


1 


- 1 


-1 


0.90 


1 


- 1 


m I 


- I 


0.95 


1 


1 


- 1 


-1 


0.95 


1 


1 


- 1 


- 1 


1.00 


1 


1 


1 




1.00 


1 


1 


- 1 


- 1 


1.05 


1 


1 


1 


-1 


1.05 


1 


1 


m 1 


■ 1 


1.10 


1 


1 


1 


-1 


1.10 


1 


1 


- 1 


■ 1 


1.15 


1 


1 


- 1 


-1 


1.15 


1 


1 


- 1 


- 1 


1.20 


1 


1 


1 


1 


1.20 


1 


1 


- 1 




1.25 


1 


1 


1 


1 


1.25 


1 


1 


- 1 


- 1 


1.30 


1 


1 


1 


1 


1.30 


1 


1 


1 




1.35 


1 


1 


1 


1 


1.35 


1 


1 


1 




1.40 


1 


1 


1 


1 


1.40 


1 


1 


1 




1.45 


1 


1 


1 


1 


1.45 


1 


1 


1 




1 .50 






1 


1 


1.50 






1 




1.55 






1 


1 


1.55 






1 




1.40 






1 


1 


1.30 






1 




1.35 






1 


1 


1.35 






1 




1 .70 






1 


1 


1.70 






1 




SUBJECT 


NUMBER 3 






SUBJECT 


NUMBER 4 










Cl 


C2 








Cl 


C2 




B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


SI 


S2 


0.50 


• 1 


m I 






0.50 


. | 


. | 






0.55 


- 1 


- 1 






0.55 


- 1 


- 1 






o.so 


- 1 


- 1 






0.30 


1 


- 1 






0.35 


- 1 


m I 






0.35 


m 1 


- 1 






0.70 


- 1 


m I 






0.70 


m 1 


- 1 






0.75 


- 1 


- 1 


- l 


- 1 


0.75 


1 


- 1 


•» 1 


- 1 


0.30 


- 1 


- 1 


- 1 


m I 


0.30 


- 1 


- l 


- 1 


- 1 


0.85 


a* 1 


- 1 


- 1 


- | 


0.35 


m I 


- 1 


- | 


- | 


0.90 


- 1 


m I 


- l 


m I 


0.90 


- 1 


- 1 


- 1 


- 1 


0.95 


- 1 


- | 


m 1 


- 1 


0.95 


1 


1 


- 1 


- 1 


1.00 


- 1 


- 1 


m 1 


m I 


1.00 


1 


1 


- 1 


- 1 


1.05 


- 1 


- 1 


m 1 


- | 


1.05 


1 


1 


- | 


- 1 


1.10 


- 1 


- 1 


- 1 


- | 


1.10 


1 


1 


- 1 


- 1 


1.15 


- 1 


- 1 


m t 


- 1 


1.15 


1 


1 


- 1 


- 1 


1.20 


- 1 


m I 


- 1 


- 1 


1.20 


1 


1 


- 1 


- 1 


1.25 


m I 


- 1 


- 1 


- 1 


1.25 


1 


1 


- 1 


1 


1.30 


- 1 


- 1 




- 1 


1.30 


1 


1 


1 


1 


1.35 


1 


1 


m I 


1 


1.35 


1 


1 


1 


1 


1.40 


- 1 


1 


1 


- 1 


1.40 


1 


1 


1 


1 


1.45 


1 


- 1 


1 


1 


1.45 


1 


1 


1 


1 


1.50 






1 


1 


1.50 






1 


1 


1.55 






1 


1 


1.55 






1 


1 


1.30 






1 


1 


1.30 






1 


1 


1.35 






1 


1 


1.35 






1 


1 


1.70 






1 


1 


1.70 






1 


1 



O 

ERIC 



77 



Table l4 (Contd) 



1.50 

1.55 

1.60 

1.65 

1.70 



SUBJECT 


NUMBER 5 






SUBJECT 


NUMBER 






Cl 


C2 








Cl 


B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


0.50 


. 1 


. | 






0.50 


1 


. i 


0.55 


- 1 


- 1 






0.55 


• | 


- 1 


0.60 


- i 


- 1 






0.60 


- 1 


- 1 


0.65 


- i 


- 1 






0.65 


1 


- | 


0.70 


- 1 


• 1 






0.70 


• l 


1 


0.75 


- 1 


- 1 


- 1 


• 1 


0.75 




1 


0.80 


- 1 


• | 


- 1 


- 1 


0.80 


1 


1 


0.85 


- 1 


- 1 


- 1 


- 1 


0.85 


1 


1 


0.90 


- i 


- 1 


- i 


- 1 


0.90 


1 


1 


0.95 


- 1 


- 1 


- 1 


- 1 


0.95 


1 


1 


1.00 


m 1 


- i 


- 1 


- 1 


1.00 


1 


1 


1.05 


-1 


1 


- i 


- 1 


1.05 


1 


1 


1.10 


m t 


- 1 


- 1 


- 1 


1.10 


1 


1 


1.15 


- 1 


i 


m 1 


- 1 


1.15 


1 


1 


1.20 


1 


• i 


m 1 


- 1 


1.20 


1 


1 


1.25 


m 1 


• i 


1 


- 1 


1.25 


1 


1 


1.50 


1 


i 


m I 


1 


1.30 


1 


1 


1.35 


1 


i 


1 


1 


1.35 


1 


1 


1.40 


1 


i 


1 


1 


1.40 


1 


1 


1.45 


1 


l 


1 


1 


1.45 


1 


1 


1.50 






1 


1 


1.50 






1.55 






1 


1 


1.55 






1 .60 






1 


1 


1.60 






1.65 






1 


1 


1.65 






1.70 






1 


1 


1.70 






SUBJECT 


NUMBER 7 






SUBJECT 


NUMBER 






Cl 


C2 








Cl 


B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


0.50 




1 






0.50 


. i 


1 


0.55 


1 


1 






0.55 


1 


- 1 


0.60 


1 


- 1 






0.60 


- 1 


- 1 


0.65 


1 


1 






0.65 


- 1 


1 


0.70 


1 


1 






0.70 


1 


1 


0.75 


1 


1 


- 1 


- 1 


0.75 


- 1 


1 


0.80 


1 


1 


1 


- 1 


0.80 


1 


1 


0.85 


1 


- 1 


m 1 


- 1 


0.85 


1 


1 


0.90 


1 


1 


- 1 


- 1 


0.90 


1 


- 1 


0.95 


1 


1 


1 


- 1 


0.95 


1 


1 


1.00 


1 


1 


m 1 


- 1 


1.00 


1 


1 


1.05 


1 


1 


1 


1 


1.05 


1 


1 


1.10 


1 


1 


1 


1 


1.10 


1 


1 


1 .15 


1 


1 


- 1 


1 


1.15 


1 


1 


1.20 


1 


1 


1 


1 


1.20 


1 


1 


1.25 


1 


1 


1 


1 


1.25 


1 


1 


1.30 


1 


1 


1 


1 


1.3 0 


1 


1 


1.35 


1 


1 


1 


1 


1.35 


1 


1 


1.40 


1 


1 


1 


1 


1.40 


1 


1 


1.45 


1 


1 


1 


1 


1.45 


1 


1 



1.50 

1.55 

1.60 

1.65 

1.70 
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Table 14 (Contd) 



SUBJECT NUMBER 9 
Cl 



C2 



SUBJECT NUMBER 10 
Cl 



C2 



B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


SI 


S2 


0.50 


• 1 


• 1 






0.50 


1 


1 






0,55 


- 1 


- 1 






0.55 


1 


- 1 






o.so 


- 1 


- 1 






0.40 


1 


- | 






0.35 


- 1 


- 1 






0.35 


1 


1 






0.70 


-1 


- 1 






0.70 


1 


1 






0.75 


- 1 


- 1 


- 1 


- 1 


0.75 


1 


1 


1 


- 1 


0.80 


- 1 


- 1 


- 1 


- 1 


0.80 


1 


1 


1 


• 1 


0.85 


- 1 


- 1 


- 1 


- l 


0.85 


1 


1 


- 1 


• | 


0.90 


- 1 


- 1 


- 1 


- l 


0.90 


1 


1 


1 




0.95 


-1 


- 1 


- 1 


- 1 


0.95 


1 


1 


1 


• 1 


1.00 


- 1 


- 1 


- 1 


- 1 


1.00 


1 


1 


1 




1.05 


- 1 


1 


- 1 


- l 


1.05 


1 


1 


1 




1.10 


- 1 


- 1 


- 1 


- 1 


1.10 


1 


1 


1 


- 1 


1.15 


- 1 


1 


- 1 


- 1 


1.15 


1 


1 


1 




1.20 


1 


- 1 


- 1 


- l 


1.20 


1 


1 


1 




1.25 


1 


1 


- 1 


- 1 


1.25 


1 


1 


1 




1.30 


1 


1 


1 


- | 


1.30 


1 


1 


1 




1.35 


1 


1 


1 


l 


1.35 


1 


1 


1 




1 .40 


1 


1 


1 


1 


1.40 


1 


1 


1 




1.45 


1 


1 


1 


1 


1.45 


1 


1 


1 




1.50 






1 


1 


1.50 






1 




1.55 






t 


1 


1.55 






1 




1.30 






1 


1 


1.30 






1 




l.«55 






1 


1 


1.35 






1 




1.70 






1 


1 


1.70 






1 





SUBJECT NUMBER 11 
Cl 



C2 



SUBJECT NUMBER 12 
Cl 



C2 



B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


SI 


S2 


0.50 


• 1 


• 1 






0.50 


• 1 


• | 






0.55 


- 1 


- 1 






0.55 


- 1 


- 1 






0.30 


• | 


- 1 






0.30 


- 1 


- 1 






0.35 


- 1 


- 1 






0.35 


- 1 


- l 






0.70 


- 1 


- 1 






0.70 


- 1 


- | 






0.75 


- 1 


- 1 


- 1 


- 1 


0.75 


- 1 


- | 


- 1 


- | 


0.80 


- 1 


- 1 


- 1 


- 1 


0.30 


- 1 


- l 


- 1 


- | 


0.85 


-1 


- 1 


- 1 


- 1 


0.35 


- 1 


- | 


- 1 


- | 


0.90 


1 


- 1 


- 1 


- 1 


0.90 


- 1 


- | 


- 1 


- | 


0.95 


1 


- 1 


- 1 


- 1 


0.95 


- 1 


- 1 


- 1 


- 1 


1 .00 


1 


1 


- 1 


- l 


1.00 


1 


1 


- 1 


- 1 


1.05 


1 


1 


- 1 


- 1 


1.05 


- 1 


1 


- 1 


- 1 


1.10 


1 


1 


- 1 


- 1 


1.10 


- 1 


- | 


- 1 


- 1 


1.15 


1 


1 


- 1 


- 1 


1.15 


- 1 


1 


- 1 


• 1 


1 .20 


1 


1 


1 


- l 


1.20 


1 


- | 


- 1 




1.25 


1 


1 


1 


- 1 


1.25 


l 


1 


- 1 


- 1 


1.30 


1 


1 


1 


l 


1.30 


1 


1 


1 


- | 


1.35 


1 


1 


1 


1 


1.35 


1 


1 


1 




1.40 


1 


1 


1 


l 


1.40 


1 


1 


1 




1.45 


1 


1 


1 


l 


1.45 


1 


1 


1 




1.50 






1 


l 


1.50 






1 




1.55 






1 


l 


1.55 






1 




1.30 






1 


l 


1.30 






1 




1.35 






1 


l 


1.35 






1 




1.70 






1 


l 


1.70 






1 
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Table Ik (Contd) 



SUBJECT NUMBER 13 
Cl 



C2 



SUBJECT NUMBER 14 
Cl 



B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


0.50 


1 


• | 






0.50 


• I 


. 1 


0,55 


- 1 


- 1 






0.55 


l 


• 1 


0*30 


- 1 


1 






0. SO 


• 1 


. | 


0*35 


1 


1 






0.S5 


• 1 


• } 


0,70 


1 


- 1 






0.70 


w 1 


• | 


0,75 


1 


1 


• 1 


- 1 


0.75 


1 


. | 


0.80 


1 


1 


- 1 


- 1 


0.80 


1 


- | 


0.85 


1 


1 


- 1 


- 1 


0.85 


1 


• | 


0.90 


1 


1 


1 


1 


0.90 


_ | 


. | 


0.95 


1 


1 


1 


• 1 


0.95 


1 




1 .00 


1 


1 


1 


- 1 


1.00 


1 


• } 


1.05 


1 


1 


- 1 


1 


1.05 


1 




1.10 


1 


1 


1 


1 


1.10 


1 


- } 


1.15 


1 


1 


1 


1 


1.15 


1 




1.20 


1 


1 


1 


1 


1.20 


1 




1.25 


1 


1 


1 


1 


1.25 


1 




1.30 


1 


1 


1 


1 


1.30 


1 




1 .35 


1 


1 


1 


1 


1.35 


1 




1 .40 


1 


1 


1 


1 


1.40 


1 




1.45 


1 


1 


1 


1 


1.45 


1 




1.50 






1 


1 


1.50 






1.55 






1 


1 


1.55 






1 .SO 






1 


1 


1.S0 






1 .S5 






1 


1 


1.85 






1.70 






1 


1 


1.70 







C2 

SI S2 



SUBJECT NUMBER 15 



SUBJECT NUMBER 1* 



Cl 



C2 



Cl 



B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


0.50 


-1 


-1 






0.50 


_ * 


m t 


0.55 


-1 


-1 






0.55 


• 1 


m 1 


0.30 


-1 


-1 






o.so 


- 1 


m 1 


0.55 


-1 


-1 






0.S5 


1 


m 1 


0.70 


-1 


-1 






0.70 


• 1 


m 1 


0.75 


-1 


1 


-1 


-1 


0.75 


1 


m 1 


0.80 


1 


-1 


-1 


-1 


0.80 


1 




0.85 


1 


-1 


-1 


-1 


0.85 


1 


- | 


0.90 


1 


-1 


-1 


-1 


0.90 


1 




0.95 


1 


1 


-1 


-1 


0.95 


1 




1 .00 


1 


1 


-1 


-1 


1.00 


1 




1.05 


1 


1 


1 


-1 


1.05 


1 




1.10 


1 




1 




1. 10 


1 




1.15 


1 


1 


1 


1 


1.15 


1 




1 .20 


1 


1 


1 


1 


1.20 


1 




1.25 


1 


1 


1 


1 


1.25 


1 




1.30 


1 


1 


1 


1 


1.30 


1 




1.35 


1 


1 


1 


1 


1.35 


1 




1 .40 


1 


1 


1 


1 


1.40 


1 




1.45 


1 


1 


1 


1 


1.45 


1 




1.50 






1 


1 


1.50 






1.55 






1 


1 


1.55 






1.S0 






1 


1 


l.SO 






1.S5 






1 


1 


1.S5 






1.70 






1 


1 


1.70 







C2 

SI S2 
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Table l4 (Contd) 



SUBJECT NUMBER 17 



SUBJECT NUMBER 18 



Cl C2 



Cl 



B 


SI 


S2 


SI 


S2 


B 


SI 


S2 


0.90 


-1 


-l 






0.90 


-1 


* 1 


0.99 


-1 


-1 






0.99 


-1 


m 1 


0.40 


-1 


-1 






0.40 


-1 


m 1 


0.49 


-1 


-1 






0.69 


-1 


m 1 


0.70 


-1 


-1 






0.70 


-1 


m 1 


0.79 


-1 


-1 


-1 


-1 


0.79 


-1 


m 1 


0.80 


-1 


-1 


-1 


-1 


0.80 


-1 


m 1 


0.89 


-1 


-1 


-1 


-1 


0.89 


-1 


m 1 


0.90 


-1 


-1 


-1 


-1 


0.90 


-1 


m 1 


0.99 


-1 


-1 


-1 


-1 


0.99 


-1 


m 1 


1.00 


-1 


-1 


-1 


-1 


1.00 


-1 


m 1 


1.09 


-1 


-1 


-1 


-1 


1.09 


-1 


m 1 


1.10 


-1 


-1 


-1 


-1 


1.10 


-1 


m 1 


1.19 


-1 


-1 


-1 


-1 


1.19 


-1 


m 1 


1.20 


-1 


-1 


-1 


-1 


1.20 


-1 


m 1 


1,29 


-1 


-1 


-1 


-1 


1.29 


-1 


m 1 


1.30 


1 


-1 


1 


-1 


1.30 


-1 


m 1 


1.39 


1 


1 


• 1 


-1 


1.39 


-1 


m I 


1.40 


1 


• 1 


1 


-1 


1.40 


-1 




1.49 


1 


1 


1 


1 


1.49 


1 


• 1 


1.90 






1 


1 


1.90 






1.99 






1 


1 


1.99 






1.60 






1 


1 


1.40 






1.49 






1 


1 


1.45 






1.70 






1 


1 


1.70 







SUBJECT NUMBER 19 

Cl C2 

B SI S2 SI S2 



0.90 
0.99 
0.40 
0.49 
0.70 
0.79 
0.80 
0.89 
0.90 
0.99 
1 .00 

1.09 

1.10 

1.19 

1.20 

1.29 

1.30 

1.39 

1.40 
1 .49 
1.90 
1.99 
1.40 
1.49 
1.70 
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placed in a lopsided 3-way table. The entry for subject number 1 in row 
.50, column SI and layer Cl is -1. This means that the subject Judged 
wider than the first element of (which has b =* ,50). An entry 
of 1 would mean that the opposite Judgment occurred. To give one more 
example from the data of subject 1, consider the entry for row 1.55> 
column S2 and layer C2, which happens to be 1. This means that the 
element of Cg with b » 1.55 (the 17th) was Judged wider than Sg . 

Analysis 

To analyze these data we will apply the countback method for con- 
structing confidence limits. Before deciding on a value for c , 
however, we should consider the way. in which we would like to use the 
confidence limits. 

In terms of an overall explanation of each subject's behavior, there 
are two natural hypotheses to apply. We may call them 2D and 3D. The 
2D hypothesis is that subjects ignore the perspective cues and simply 
match the widths of the projected image. In the 3D hypothesis, it is 
assumed that the subjects treat the images as though they were the 
objects which generated them. We will interpret these hypotheses as 
. giving locations for the 50$ point in each of the four series. 

The instructions attempted to direct the subjects toward 3D, but 

* 

there were subjects who spontaneously reported their interpretation in 

4 

terms of 2D. On these grounds alone, we might expect to find some subjects 

1 

of each type. 



Subject 18 was one of these. 

o 
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An interesting question arises as to whether there are any subjects 
for whom neither 2D nor 3D is applicable. Preliminary observations 
suggested that this would be the most common situation. Consequently, 
the comparison series were chosen to be most appropriate to responses 
representing a compromise between 2D and 3D. In fact, perfect 3D 
behavior would require a subject to report every element in and 

as wider than either or Sg • On the other hand, a perfect 2D 

subject would find and wider than every element in and 

wider than all but the largest five in C ^ . The effect of this choice 
of series is that confidence intervals for such subjects may be open- 
ended on one side or the other. 

Returning to the discussion of confidence limits, there are many 
ways they could be used to test these hypotheses for each subject. The 
way used here is to reject a given hypothesis for a subject if at least 
three out of the four predicted 50$ points do not fall between their 
appropriate confidence limits. 

If the confidence coefficient for each confidence interval taken 

separately is 1 - P , what is the probability that at least three of 

the four fail to contain their 50$ point for at least one of N subjects? 

For any given subject, the probability of this occurrence is just the 

probability that none, or only one, contains its 50$ point, or 
k z 5 k 

P + 4p^(l - P) - 4 p^ - 3P • The probability of this happening for at 

3 4 

least one of the N subjects must be at most N(^P^ - 3P ) . (This 
would be the exact value only if occurrences for individual subjects were 
mutually exclusive events.) 
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In the present case, with N = 19 , confidence limits were found 
using c = 5 . (These are reproduced in Table 15 •) For this value of 
A c , we know that 1 - P is greater than .875* Thus, the probability of 

three or four 50$ points falling outside their confidence intervals for 

> 

at least one of the 19 subjects is less than .13^521. In view of the 
* conservative nature of the lower bound on 1 - P , this is quite a 

reasonable value, (it may be noted that, if 1 - P were in fact .95 for 
each subject, the probability would drop to .0091M*.) 

Looking at Table 15, we can see that possible 3D subjects are 'J, 8, 
10, and 13. (Subject 8 is actually subject 7 trying again.) 3D must be 
rejected for all the others. Possible 2D subjects are 3 , 17 > and 18. 
Consequently, subjects 1, 2, k, 5, 6, 9, 11, 12, 14, 15, 16, and 19 are 
neither 2D nor 3D, but somewhere in between. 
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Table 15 

Countback Confidence Limits for the 50$ Point of Each 
of the Four Series for Each Subject; Given in 
Terms of the Comparison Widths, with c = 3 



Subject 


S vs C 
1 1 


S 2 vs C ± 


S 1 vs C 2 


S 2 VS °2 


1 


( . 50 , 


1.00) 


( -00 , 1.00) 


( .85, 1.30) 


(1.00, 1.35) 


2 


( . 70 , 


1.05) 


( .75, 1.10) 


(1.10, 1.45) 


( 1 . 05 , 1 . 40 ) 


3 


( 1 . 20 , 


" ) 


(1.20, °= ) 


(1.15, 1.50) 


( 1 . 20 , 1 . 55 ) 


4 


( -°° , 


1.05) 


( .75, 1.10) 


(1.10, 1.45) 


O 

• 

H 

•V 

LPv 

O 

• 

H 


5 


( 1 . 05 , 


1.4o) 


( .90, 1 . 40 ) 


(1.10, 1.45) 


(1.10, 1.45) 


6 


( -°° , 


.85) 


( .50, .85) 


( .95, 1.30) 


(1.00, 1.35) 


7 


( - 00 , 


. 70 ) 


( , 1 . 00 ) 


( , 1.30) 


( .85, 1.20) 


8 


( “°° , 


• 90 ) 


( -°°, 1 . 05 ) 


( , 1.30) 


( .85, 1.30) 


9 


( 1 . 00 , 


1 . 55 ) 


( . 90 , 1 . 35 ) 


(1.10, 1.45) 


(1.15, 1.50) 


10 


( -°° , 


.65) 


( -”, - 75 ) 


( , 1.00) 


( .75, 1.25) 


11 


( . 70 , 


1 . 05 ) 


( .80, 1.15) 


(1.00, 1.35) 


(1.10, 1.45) 


12 


( .85, 


1 . 30 ) 


( .85, 1 . 55 ) 


(1.10, 1.45) 


(1.05, 1.45) 


13 


( -°° , 


. 75 ) 


( , .85) 


( .75, 1.20) 


( .75, 1.15) 


14 


( -°° , 


1 . 05 ) 


( .80, 1.25) 


( .90, 1.25) 


(1.10, 1.45) 


15 


( .60, 


• 95 ) 


( .60, 1.05) 


( .85, 1.20) 


( .95, 1.30) 


16 


( .50, 


.85) 


( . 65 , 1 . 00 ) 


( .85, 1.20) 


( .95, 1.30) 


17 


( 1 . 10 , 


1 . 45 ) 


( 1 . 20 , » ) 


( 1 . 15 , 1 . 50 ) 


(1.25, 1.60) 


18 


(1.25, 


" ) 


( 1 . 25 , » ) 


( 1 . 30 , 1 . 65 ) 


(1.25, 1.60) 


19 


( .65, 


1 . 05 ) 


( .75, 1.10) 


( 1 . 05 , 1 . 40 ) 


(1.15, 1.70) 
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Recent Applications of Maximum Likelihood 
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